Monday, May 11, 2015

PHP Elastic Driver

Welcome back.

Today we will be looking at how to link Elastic Search into your PHP project. Lucky for us Elastic already has an well put together composer project. This project with all its documentation can be obtained here.

So for the SQL boys that have not played with No-SQL databases before I will go through some of the basic queries.

First lets start with the simple SELECT with a LIKE or a MATCH AGAINST in mysql that is.

In Elastic I have a index with the following fields

Index: TestDB
Type: TestRecords
Mappings: id,name,description,datecreated

In Mysql terms
Database: TestDB
Table: TestRecords
Columns: id,name,description,datecreated

Now that is the very basic breakdown if you want to go into this further you can start looking at how the mappings are analyzed for better performance.

Now the PHP part as we all know how to do the MySql SELECT.

Start a connection to Elastic
$elastic = Elasticsearch\ClientBuilder::create()->setHosts(['yourip:9200'])->build();

Set the search params
$params['index'] = 'TestDB';
$params['index'] = 'TestRecords';
$params['body']['query']['match']['description'] = 'phrase search';

This will return a set of results that  will be list in terms of score. The data record that matches the best will have the highest score. This is very fast and allows you to order the data so your client sees the most relevant items first.

I will say that the documentation is pretty go but if you have any questions with regard to ranges or aggregations just drop a comment.

Friday, May 8, 2015

Elastic Search Integration

So before we continue with the Phalcon integration lets have a little fun. Our current system makes use of a Mysql db soon to be a Maria db for its transactional database. Now we are starting to reach 60 million records in certain tables.

Now I tell you one thing even with the beast of the server we have when you start text searching on 60 million records things go haywire.

So how do we resolve this issue?

After a developer in our team wasted many hours looking at different offerings we decided on using Elastic Search now called Elastic. I was a little unsure at first but after watching the video of Netflix's integration using Elastic I was sold. Of course you cant go by what other people say because every ones system is different. So I did my own test on my local system and here are the results.

Full phrase search on my local mysql on 500 000 records 30-50 seconds.
Full phrase search on my local elastic on 500 000 records 0.02 seconds no variation.

Okay now I am totally sold so how do we make use of this awesome system?

To start with we consolidate all our financial tables into one index(type of table in elastic) and now where we used to have a long statement with many inner joins that took ages to run and added load to our database we now have a 10 second pull from Elastic.

This is just one application we have many others that will soon be implemented.

In my next  post I will go deeper into the setting up of Elastic setup and the PHP query driver.

Phalcon Smarty Integration

So after a lot of late nights by one of my senior developers we now have Phalcon neatly integrated into our code base.

From the beginning of the project we have been making use of the Smarty templating engine and although Phalcon has its own (Volt) due to the team being used to Smarty and in my opinion Smarty being more powerful we have opted to continue with it.

In order to integrate Smarty into Phalcon we customised the class that incubator created it can be retrieved here  . This class extends the Engine class from Phalcon which later uses dependency injection to integrate Smarty into Phalcon,

So now that we have that class integrated we need to make use of it. In our public folder of Phalcon we setup a kernel file for each one of our sites.

    require dirname(__DIR__) . "/configuration.php";

     * Read the configuration
    $config = include  WEBS_DIR . "/config/config.php";

     * Read auto-loader
    include WEBS_DIR . "/config/loader.php";

     * Read services
     * @var \Phalcon\Di $di
    include WEBS_DIR . "/config/services.php";

     * Read services
    include WEBS_DIR . "/config/routes.php";

     * Handle the request
    $application = new \Phalcon\Mvc\Application($di);


    // Register the installed modules
            'website'  => array(
                'className' => 'Axxess\Website\Module',
                'path'      => WEBS_DIR . 'class.Module.php',

    echo $application->handle()->getContent();

catch (\Exception $e)
    echo $e->getMessage();

So now that we have setup our kernel we have to create the Smarty setup in the services.php in out site's folder.
Firstly instantiate the Phalcon dependency injector.

 * The FactoryDefault Dependency Injector automatically register the right    services providing a full stack framework
$di = new FactoryDefault();

then we inject the Smarty engine

$di->set('view', function () use ($config)
    $view = new View();


            '.tpl' => function(View $view, $di)

                $smarty = new SmartyEngine($view, $di);

                $smartyInstance = $smarty->getSmarty();
                    ->setCompileDir(WEBS_DIR . 'viewsCompiled' . DS)
                    ->setCacheDir(WEBS_DIR . 'viewsCache' . DS)
                    ->setConfigDir(WEBS_DIR . 'viewsConfigs' . DS)
                    ->addPluginsDir(SMARTY_DIR . 'plugins' . DS . 'axxess' . DS . 'website');

                return $smarty;

    return $view;
}, true);

This now allows us to access smarty like we would of Volt in the view variable in the Phalcon controllers.

The dawn of Phalcon

So now that we had burned our hands with Symfony2 we decided to do some proper research. After many cups of coffee we managed to limit the available frameworks down to 3.
  1. The good old Code Igniter
  2. The new kid on the block Phalcon
  3. The native php framework addon YAF
All of these frameworks delivered decent speed and low memory usage.

Now which one do we choose?
Here is a list of criteria:
  • Community support
  • Developer support
  • Extensions
  • Ease of use
  • Ease of integration 
  • Documentation
First to go is YAF:
  • Pro's
    • Very fast
    • Built using PHP
    • Built by the creators of PHP
  • Con's
    • Almost no documentation
    • Development seems to have stopped
Second to go is Code Ignitor
  • Pro's
    • Fast
    • Built using PHP
    • Full intellisense
    • Huge community support
    • Loads of extensions
    • Very good documentation
  • Con's
    • Now owned by a School
    • Latest update hasnt made use of new PHP features
    • Community seems to be moving to Laravel
So we are finally left with the PHP & C love child whose name is Phalcon.
At first I was heavily against the idea of a C based framework in a PHP project which just sounds wrong. After a bit more research it seems that its the way of the future. The reason I say this is it seems that other older frameworks seem to be moving in this direction. Here is an interest article that might help with your decision PHP vs C

So how does Phalcon work?

The low level framework is compiled C code which is available if one is brave enough to fiddle.
Next is an intermediate layer name Zephir that allows you to build extensions for the framework that compile into native C.
Next there is a PHP interface this allows the IDE to have full intellisense which trust me is a big plus.

So the team now knew all the details on the 3 frameworks and it was time to vote. Phalcon won by a landfall.

More Phalcon to follow...

Thursday, May 7, 2015

Maturing php system

Let me give you a little background before going into detail.

When I first started on the php system I was a junior c# and didn't know the first thing about how to setup a php environment or what php was capable of. So the best way to get going is just to jump straight in.

At that time we had a perl based system which was failing up to 7 times a day and the c# replacement project that I had been part of had failed due to mis-management. The directors happened to see my php clone of the current perl system although it was in its starting phase with lots of messy classes two more juniors were hired to help me complete this php project and a decent length of time later we were ready, in our eyes that is, to go live. Messy code and all we managed to go live but now we were left with a totally non scalable code base.

So what is the first step to making coding quick, easy and clean in php?

Frameworks yes but which ones as the are so many to choose from. So our sister company currently uses symfony 2. One of our team members happen end to know symfony well so I gave him the go ahead to integrate symfony plus doctrine into our system. Once it had been integrated with a bit of hassle I asked him to run a few speed tests. 

We ran four tests and I list them in their speed order
Legacy with doctrine
Legacy with pdo 
Symfony with doctrine apc cache
Symfony with pdo and apc cache 
Symfony with doctrine
Symfony with pdo

The results were amazing.
The fastest as listed above was between 8-10x faster then the slowest and this is not a hello world benchmark. It was a simple db query on a no load db. The memory usage on symfony was also massive. So where do we go from here? To be continued.....

PS I will send a link to the full tests later.