Archive | Web Design RSS feed for this section

A Header Apart

For the past year or so, the header on A List Apart has slowly driven us a bit nuts. It is hands down one of our favourite websites, but we have never been able to quite understand the design derision around why the header was squashed up off the top of the page:

The Live Site:

However, a friend of ours has answered our prayers and put together a fun little extension for Google Chrome to adjust the layout and push the header banner down to a more attractive location. Check it out at the Chrome Web Store.

After enabling the extension:

Food for thought

What other websites could do with some tweaks? Is there scope for a crowd-sourced plugin to fix all the annoying design choices out there in the web? Would you contribute? How would you feel if your website was being modified by a plugin? Would you take it as good advice and update your own design?

Getting to grips with Laravel

Laravel is a great PHP framework for developing anything from simple websites to complicated applications. We are currently using it on a rather complex system for one of our clients and we are so far really pleased with it.

With the introduction of version 4, there is incredible flexibility built into the framework to allow us to control everything from Navigation URLs, data base management, security and user account and how we manage the templates for the user interface.

Still early days, but so far, so good. If you have never used Laravel, but are looking for the next tool to try out, we’d certainly recommend giving it a shot: http://laravel.com/

Ready for Hack Day

We’re running our first internal hack day tomorrow. Basically, the team each has 24 hours to build something, anything, using whatever technology they can get their hands on. We’ve got some people looking at various things, including Node.js, Angular.js, NoSQL databases and the use of gameification in our web systems. It’s looking to be a very exciting and fun day. We’ll try and write up some of the more interesting things we produce.

Google Web Designer

Google have today launched a new tool, in BETA of course, for creating HTML5 adverts. The Google Web Designer will allow the creation of ads and campaigns that are accessible to all level of user and that will run on desktops, smart phones and tablets. It will be interesting how far they push the capabilities of the web designer tool, and whether it will become a fully fledged web design tool, or whether they will remain in the ad designing world, but so far it’s looking pretty cool.

Time is running out for IE8

At long last we are nearing the end of Windows XP and Officer 2003. This is surely good news. For too long businesses around the country have been holding on to Windows XP as their primary operating system. Now in itself, XP is not a bad operating system, but for us web developers, it is the course of many a night without sleep.

Windows XP supports Internet Explorer 8 as the latest version. Internet Explorer is 4 and a half years old already. Come April 2014 it will be five years old but we shall finally all get to wave it good bye, and get on with building websites for the modern browser.

If you are STILL using Windows XP and IE 8, I suggest you take action sooner rather than later. Switch to Chrome or Firefox if you cant upgrade your copy Windows just yet, and don’t forget to tell your IT department to get a move on with the upgrade. Web browsers should never really be more than 1 year old in this day and age – certainly not 5!

My Photo Folio

We’ve started working with MyPhotoFolio to develop a custom content management system. This CMS will form the backbone of their business, which plans to allow photographers to create their very own website. It’s early days right now, but we’re very excited to be involved with it.

Concrete5 “Enterprise Ready”? Not entirely…

We’ve written a lot about Concrete5 in Head Energy’s blog, with significant effort spent on documenting our high level AWS architecture for a large Concrete5 Sites. However, going back one step more (and stepping over my hate for the phrase “enterprise ready”, which seems to be synonymous with “bloated, cryptic and slow as hell”), we’re forced to ask today: is Concrete5 really an “enterprise ready” system?

Our project, built on v5.6.1, probably has 50k+ lines of code these days, and one of the ever increasing worries during our development has been that of slow page loads. We’re seeing un-cached page load times creep up from their early renders of 1s-2s to a current average of 4s-6s, and it’s getting worse linearly with the complexity of the page. The linearity suggests we’re simply introducing a greater workload and have only ourselves to blame, but after just a few minutes of looking for high load anywhere in our architecture I was at a loss, until I did the bog standard check for database queries.

The number of queries Concrete5 generates when using it’s own API is utterly staggering a landing page deemed moderately complex generated precisely 6882 queries to render from start to finish.

The number of queries Concrete5 generates when using it’s own API is utterly staggering a landing page deemed moderately complex generated precisely 6882 queries to render from start to finish, and the agent wasn’t even logged in! This isn’t a problem on a developers machine, where the latency between their MySQL layer and the web server is effectively 0. When we add a 0.5ms latency (think AWS load balancers, traffic managers or just networking!) the delay from latency balloons to 3.441 seconds in additional page load time.

What’s worse is we’ve taken some significant steps to reduce the number of Page List blocks in use on Concrete5, simply because we already knew it was ridiculously heavy on the database with only a few dozen pages in play. We wrote an override which completed what Concrete5 was achieving in the Page List from hundreds of statements in just one (without permissions however, but more on that in a moment).

Finding out the number of database queries Concrete5 generates per page

Concrete5 comes packing the adodb abstraction layer, which has a nice inbuilt function to log all database queries into a table. To make use of it you’ll probably have to create the logging table manually (we did) the statement for which is:

CREATE TABLE IF NOT EXISTS `adodb_logsql` (
  `created` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  `sql0` varchar(250) NOT NULL DEFAULT '',
  `sql1` text NOT NULL,
  `params` text NOT NULL,
  `tracer` text NOT NULL,
  `timer` decimal(16,6) NOT NULL DEFAULT '0.000000'
)

Once created switching on the logging function within Concrete5 is as simple as calling

Database::setLogging(true);

You can use this around particular operations in your code, just to take a peek, or just as a quick solution throw it right at the bottom of:

concrete/core/libraries/database.php

Now, every time you reload a page in Concrete5, the adodb_logsql table will fill up with the queries generated to make that page load. We’ve confirmed our suspicions that the Page::getBy* methods are kicking out a significant chunk of our statements, and plan to remove as many of these as possible over the next few days in place of customized SQL.

Is Concrete5 Enterprise Ready?

If your environment is very cache friendly, or you don’t have too many pages (arbitrarily e.g. 250+), then yes you’ll love the entire journey without the need to get very technical. If this isn’t the case, then you’ll quickly need a set of proficient developers who are happy to replace chunks of Concrete5′s native functionality with more efficient alternative routines (namely less SQL statements!).

That said, we’re still hugely enamoured with our choice of platform, and do not believe any other system offers a better ratio of user friendliness, flexibility, scalability and out of the box CMS features, at least not yet.

Bullet proof JavaScript for CMS

A web content management system (CMS) allows editing, styling and publishing of content for a website from a single piece of software. Some common CMS choices include:
  • WordPress
  • Concrete5
  • Joomla
  • Drupal
The main two ways of editing the content in a web based CMS; In page, and off page.

In page, or in-line editing, allows a user to view their changes as they happen in the context of the web site design. Off page editing has a separate section for editing the content so viewing it on page requires the changes to be either previewed, or even saved first.

Each system has its advantages, but where some in page systems really fall down is with regards to JavaScript. JavaScript code executes directly in the web browser to add powerful functionality to a website. The major downside to JavaScript is that it can easily be broken by inexperienced developers and once a single error occurs, the remaining JavaScript fails to execute. 

An example of this happened to us recently when we were adding a third party block to a client’s Concrete5 website. The developer of the block had an intermittent error in their JavaScript code. Once we added the block to a page, we could no longer edit the page to remove the block! The Concrete5 user interface is reliant on JavaScript code, and the error in the third party block was causing the Concrete5 code not to run. Fortunately, we had the know-how to remove the bad block: For those that are stuck on this, you can often revert your pages to an earlier version (that is, a version that doesn’t contain your troublesome block) by accessing the Dashboard > Full Sitemap, clicking on the page in trouble, and selecting Versions.

The really annoying thing, is that this type of problem is incredibly easy to avoid by writing better JavaScript in the first place. Simply detect errors in each code block and deal with them accordingly. This is what we ended up doing with the third party block in this case. 

I want to implore anyone who works with a CMS that allows In Page editing to be more diligent with their JavaScript code and catch and handle exceptions at all times. Just like we do.

PHP on EC2 (AWS) with Multi A-Z and Multi Region (Part 4)

Looking for Part 1? See: http://www.headenergy.co.uk/2012/10/php-on-ec2-aws-with-multi-a-z-and-multi-region
Looking for Part 2? See: http://www.headenergy.co.uk/2012/10/php-on-ec2-aws-with-multi-a-z-and-multi-region-part-2
Looking for Part 3? See: http://www.headenergy.co.uk/2012/11/php-on-ec2-aws-with-multi-a-z-and-multi-region-part-3
 
 

A Quick Intro

There seems to be quite an appetite for this mini-series on the web, so I’ve sped up the writing process a little in the hope of mind dumping what remains of this content before people lose interest.

In my recent posts I’ve discussed the problems of deploying a popular PHP CMS into the AWS cloud hosting platform, the options one might consider in spreading out the database tier in a non-split environment, and finally how we can best leverage our choice of Galera. If you’re buying what we’re selling here, then you’ve got a multi-master set up, with good resiliency to zone, and even region, failure at the database tier. This post will go into more detail regarding our chosen configuration, including some performance notes so we’re better informed of the power of this deployment.

Our Configuration Choice

For those that don’t recall, part 3 of this mini-series handled the different configurations open to us with regard to Galera. Our recommendation reads:

We’ve chosen to go with the DBMS tier cluster along with a distributed load balancer as we cross into different regions. This is primarily due to our platform being EC2, as it provides a resilient, reliable and cost effective load balancer, removing the worry of having a single point of failure. This also simplifies the deployment as more database servers are removed or added to the pool, as we only have to configure a single load balancer.

In reality, we don’t know what our load will be precisely, so we’re going to go with the straightforward minimum 3 production servers for our tests, along with 1 additional server on standby.

Benchmark tests (using sysbench with 15 tables of 2m records each) show the potential Transactions Per Second (TPS) and latency for a Galera cluster on EC2; m1.large instances (7.5Gb RAM, 2 cores). This shows that a 4 node Galera cluster on EC2 can support almost 1000 transactions per second with 110 concurrent users. With no serious degredation of latency. As a comparison, the native NDB cluster benchmark with the same tables saw latency increase to in excess of 600ms for the same 110 user concurrency. The following images show our results in greater detail:


As the system scales out it may well prove more efficient to move to the aggregated stack cluster (discussed in the previous post). After examining the the stats and performance of Concrete5, we’re confidant that a single database server can serve multiple application stacks. Particularly poignant – given we know we’ve got a ceiling on the number of Galera nodes available to us in our cluster – we can easily scale out many application stacks radio before we focus on scaling MySQL.

 

The Initial EC2 Configuration

Based on our assessment above, and assuming use of EC2, we’re going to use the following configuration:

  1. 1 VNC per site (though this is mandated by AWS)
  2. 3 Large EC2 instances running Galera, with an extra on standby.
  3. 6 EBS volumes using RAID0 for the data files and one EBS for the server application and O/S.

In future posts I’ll outline the full deployment structure we’ve chosen, including GlusterFS, a NAT entry point node, a DNS pair, Load balancer(s) (HA Proxy), underpinned by Puppet for very flexible deployments. We’re also likely to use either ZXTM (in conjunction with the Amazon AWS API) or Elastic Load Balancing to achieve automatic scaling in our deployment.

PHP on EC2 (AWS) with Multi A-Z and Multi Region (Part 2)

Looking for Part 1? See: http://www.headenergy.co.uk/2012/10/php-on-ec2-aws-with-multi-a-z-and-multi-region
Looking for Part 3? See: http://www.headenergy.co.uk/2012/11/php-on-ec2-aws-with-multi-a-z-and-multi-region-part-3
Looking for Part 4? See: http://www.headenergy.co.uk/2012/11/php-on-ec2-aws-with-multi-a-z-and-multi-region-part-4
 

A Quick Intro

In my last post I discussed the need to deploy a popular PHP CMS into the cloud which leveraged the scaleability and multi-az / multi-region redundancy offered by the excellent AWS. One of the first stumbling blocks to spreading the database load for our CMS (Concrete5 – which sure does generate a lot of queries for a page load! – ) was the inability to elegantly split our read / writes and send them to different MySQL nodes. After sifting through a few inadequate solutions, we settled on Galera as our solution.

Galera

Galera is a true multi-master, synchronous replication cluster for MySQL based on the well used InnoDB storage engine. Users can deploy the Galera cluster locally in LAN environments, as geo-clusters over the WAN or as a virtual cluster in cloud hosting platforms. Even better, Galera is offered as an Open Source software solution. It can be downloaded freely from http://www.codership.com.

The marketing blurb states:

“Galera Cluster is deployed by users who need highly available MySQL database back-ends with very fast fail-over time, consistent databases with no data loss and reduced investment in high availability architectures. Galera Cluster is ideal for business critical applications like Social networks, Gaming platforms, Telecom/VoiP solutions, Media and entertainment sites, Business Software as a Service (SaaS), Platform as a Service(PaaS), ERP systems, web-shops, e-commerce solutions or similar critical applications.”

Benefits

Multi-master replication means that all slave nodes can be used as the master at any given time, that is to say we can write to any node in the pool indiscriminately. Unlike the single threaded nature of MySQL native replication which is liable for bottlenecks, Galera runs on a parallel replication approach which improves slave lag significantly. Applications connecting to any server in the cluster can be confident they are reading and writing to the same stateful dataset.

As all nodes communicate to all other nodes, transmitting writes as appropriate, we can forget about the read / write splitting otherwise required by Concrete5 for MySQL replication.

Galera also supports WAN clustering and synchronous replication within that. Unavoidably, and understandably, there may be short periods of lag due to the network round-trip-time required by the synchronous state. Our testing shows that the round trip between Singapore, Ireland and East-Cost U.S is still well within the milliseconds rather than seconds.

For those of you running a slightly tighter applications, Galera also assigns a Global Transaction ID for all replicated transactions, therefore transactions can be uniquely referenced in any node.

Galera also facilitates automatic node joining. The cluster chooses ‘donor’ for the ‘joiner’.

The path has been well trodden on EC2, Galera has been well tested and deployed on the Amazon EC2 environment already.

Limitations

To start the cluster, Galera must have a minimum of 3 nodes in the cluster to operate effectively. As Galera will try and promote any server to a master server, a cluster of less (e.g. 2)  would suffer from a ‘split brain’  scenario wherein both servers are alive but unable to communicate to each other. With 3 servers this ‘split brain’ scenario cannot happen, so long as two servers continue to communicate. To extend this, an odd number of nodes is recommended as an application scales up.

The number of Galera nodes in the pool should not exceed 10, the mathematics are sound, if a little complicated, and can be seen on Codership here: http://www.codership.com/content/multi-master-arithmetics.

As mentioned, replication only works with the InnoDB storage engine, writes to other table types will simply not be replicated. We are using this “limitation” to easily remove replication for tables we know we don’t need to see replicated, e.g. non-critical logging tables. Rest assured however, DDL statements are replicated in statement level, and so any changes to mysql.* tables are replicated. We can safely issue:CREATE USER… or GRANT… but issuing: INSERT INTO mysql.user… would not be replicated. By their nature, non-transactional engines cannot be supported in multi-master replication.

Concrete5 requires some MyISAM tables for its text-search facility, specifically PageSearchIndex. it is relevant to note that MySQL 5.6 supports text searching within InnoDB tables (http://blogs.innodb.com/wp/2011/12/innodb-full-text-search-in-mysql-5-6-4), Galera will be releasing an updated version soon to incorporate this. In the interim, we’ve removed these indexes and moved to InnoDB regardless. In our case, we’re already deepening the functionality of the search engine by incorporating SOLR, and it has no impact on our existing project roadmap.

The next blog post will cover the various cluster configurations we have open to us on EC2 and, time permitting, some other general considerations for EC2, such as disk io, disaster recovery and more.