Go FAST - Speed up the Agile Platform compilation time by 30%

Go FAST - Speed up the Agile Platform compilation time by 30%

  
Hi everyone,

I'll share with you today a way to speed up your development efforts in the Agile Platform, by making your developers (even) more productive!

The deployment mechanism in the Agile Platform is simply amazing - one click gets you a fully running application with all the stack components (UI, Services, and Database) updated in a remote environment. Because it is fundamental to development, most people spend a significant time just publishing their applications.

In the spirit of continuous improvement, we set out to find a way to improve the performance of this process in a recent project. The Platform already does a great job at generating all the code, but we wanted to make it even faster.

After exploring several techniques, we settled on a technique that decreases compilation times by around 30% in our test environment! Over the course of a project, this amounts to hours of developer time that you can get back. The trick we used was fairly simple - install the Platform in a RAMDisk. RAMDisks are virtual hard drives mapped to RAM, which makes them tremendously faster. Because the Platform needs to generate a significant amount of files during the publishing process, hard drives can become a bottleneck to the process. If everything is in memory, access times are much faster and the whole process gains from it.

Here are the general steps necessary to implement this:
  1. Get some serious RAM. In this project we dedicated 16 GB of RAM to this RAMDisk. If you're thinking this is expensive, think back to how much time your developers can save :)
  2. Install any RAMDisk software. We have used Dataram RAMDisk but you should be able to use any other software that achieves this goal. A good performance benchmark can definetely help you chose
  3. Setup a new RAMDisk with enough space to hold the Platform installation. If you already have the Agile Platform installed, this would give you a good indication of how much memory you'll be needing. If not, I recommend you start with at least 8 GB. Important - Set the RAMDisk to periodically save the contents to a file in the disk. Additionally, this should also be automatically done on shutdown. This will keep you from having to reinstall the Platform in case you need to restart the server. Note that not all RAMDisk software packages allow you to do this, but I highly recommend you get one that does
  4. Install/Reinstall the Platform in that new hard disk. Remember to follow the checklist to make sure you don't miss anything
  5. Republish your entire factory
From this point on you should be ready to go! If you do go ahead and effectively measure the difference in your environment, make sure you post it here so that we know whether the technique works for everyone.

Have fun and fast coding!
-Goncalo
I have a suspicion I know where the inspiration for this came from... :)

All in all, good tip!

J.Ja
Hey Justin,

The original idea came from a blog post about moving Temporary ASP.NET files into a RAM Disk. We tried that but with limited success, and thought we'd go extreme an install the whole Platform in such a disk. I can't remember exactly what blog this was in, otherwise credit would have been given :)

I was curious about your comment so Googled RAMDIsk and your name, and I guess you are refering to this article? If you found other ways to make it even faster, I'm the first one in line to know about it!

Best regards,
Goncalo
Hi Gonçalo,

Did you also test with an SSD for the platform, maybe two SSD's in RAID 0, just to test the performance improvements?
Just curious for the results,

Remco
Remko,

We did not have easy access to an SSD drive so we didn't test. I'd be curious to benchmark the difference to this solution... Do you have such a setup?

Best regards,
Goncalo
Hi Gonçalo,

I'm also not so lucky to have a setup with SSD, but I'm curious if this would also shorten the compile-time.
"In-Memory Technology doesn't pay off"
Check this article. http://www.computerworld.com.pt/2010/11/24/%E2%80%9Ctecnologia-in-memory-nao-compensa%E2%80%9D/
But at the same time SAP is investing in in-memory technology: http://www.computerworld.com.pt/2010/12/15/sap-disponibiliza-tecnologia-in-memory-hana/

It's in Portuguese, sorry to the non Portuguese speakers, but you can use the translations tools available.

 
Miguel,

My point was to install the Platform in memory and not the database. The article you mention primarily talks about data and not just plain old application files. Yes, SSD would be good but this is a faster, more accessible implementation IMHO. 

That being said, in-memory technologies clearly have their place in scalable systems - think memcached! It just depends on the usages and what you want to do with it...

Thanks for posting,
Goncalo
Totally agree with you Gonçalo.
I just wanted to share those articles regarding in-memory technologies that I came across at the same time as your post.
We are using both SSD and IncrediBuild which gives us ~10 times faster compilation.
IncrediBuild allows each of our build machines to scale up and become a 200 cores machine by utilizing any idle core across the network while the SSD makes sure the Initiating machine's I/O is fast enough to handle the workload.
dima ext wrote:
We are using both SSD and IncrediBuild which gives us ~10 times faster compilation.
IncrediBuild allows each of our build machines to scale up and become a 200 cores machine by utilizing any idle core across the network while the SSD makes sure the Initiating machine's I/O is fast enough to handle the workload.
 Hi,

Can you explain more about your enviroment with the SSD and the incredibuild?

Tnks,

 

If you want to run your production server in ram, this might help http://www.superspeed.com/servers/ramdisk.php
(Not tested, if you used this before, do provide a review - is it any good?)