Guide to disk space usage and control on OutSystems Platform servers

Guide to disk space usage and control on OutSystems Platform servers

  

This article was moved to an entry in the Support Knowledge Base. You can find the updated content at www.outsystems.com/goto/disk-space-usage-guide.

The original post was kept for reference reasons, but is no longer certified and should not be used as the reference material.


Hi all


Often I get questions about the disk space usage under the Platform Server installation directory, mostly due to disk space limitations that exist on the Agile Platform servers, and usually in search for ways to control and manage that disk space usage.


Knowing the commonly largest directories


First, let me clarify on the most commonly growing folder locations under the Platform Server installation directory, in the context of an Agile Platform server installed with the .NET stack version of the platform. Each of these folder locations can reach to several tenths of GB, depending on the factory size, and the purpose of the environment and folder.


On the table below, you'll find detailed information about what it's stored on each of the folders that usually take more disk space, along with the purpose of those folders and in which server profiles and environment types we can find them using up disk space. For this table do take into consideration the following server profiles:

  • Controller: A controller is the server that is responsible for the 1-Click Publishing process, and we assume here that is it's sole purpose, meaning that is a pure controller, without the Front-End server role.
  • Front-End: The Front-End server is the server responsible for the runtime execution of the OutSystems applications, and thus, were the applications are deployed (either in the public or personal areas).


Type of contentDescriptionLocationIn Controller?In Front-End?Environment type
 Full Compilation CacheAll generated source code (.oml, .cs,.csproj, etc) files and all application release binary files (.dll, .aspx, .asmx ,etc), as well as all application resources files (.js,.css,.png,.gif, etc) in a full compilation. <platformserver>\share\<espacename>\fullYesNoAll
Release application files package (zip package)<platformserver>\share\<espacename>YesNoAll
 Partial compilation cacheAll generated source code (.oml, .cs,.csproj, etc) files and all application release binary files (.dll, .aspx, .asmx ,etc), as well as all application resources files (.js,.css,.png,.gif, etc) in a partial compilation for each developer that runs/debugs the espace. <platformserver>\share\<espacename>\partial\<developer>YesNoNon-production only (unless developers do debug on production)
 Release Application FilesAll release application files, deployed to be loaded by the Front-Ends application server (IIS). These files include all application files for this espace (.dll, .aspx, .asmx, .js, .css, images and other included resources). <platformserver>\running\<espacename>.<versionid>NoYesAll
 Personal Test Area FilesAll release application files for a specific version that a specific user is running/debugging. These files include all application files for this espace (.dll, .aspx, .asmx, .js, .css, images and other included resources). <platformserver\test\<espacename>\<developer>NoYesNon-production only (unless developers do debug on production)
 Temporary Asp.Net FilesThis files are automatically generated by the .Net Framework, and not the Agile Platform, but they are a part of every ASP.NET application.C:\windows\microsoft.net\framework\
v2.0.5.0727\Temporary Asp.Net Files\<espacename>
Or
C:\windows\microsoft.net\framework64\
v2.0.5.0727\Temporary Asp.Net Files\<espacename>
NoYesAll


A server can have both roles, which in that case, it would have both server role contents. Usually, the non-production environments use more disk space then production, mainly due to the number of developers, and due to the personal test areas for debugging.


What folder contents can be deleted?


Well, ultimately, base on the architecture of the Agile Platform where the applications are defined by single OML files and extensions, any folder content can be deleted and then recovered through internal processes, like 1-Click Publishing. However, as you might be aware, this takes time, and it's not recommended for the most environments, so let me elaborate on the side-effect of deleting each one of the presented contents:


Type of contentSide Effect of Deleting itHow to recover it
 Full Compilation CacheThis compilation cache is used when 1-Click Publishing or Debugging espaces with references. The Platform Server will fetch the producer references from this location. If they aren't here, compilation errors or runtime errors on the consumer espaces due to broken references may occur.One needs to do 1-Click Publishing of the producer espaces to rebuild the producers compilation cache. The best way is to build an All Content solution in Service Center and publish it.
 Partial compilation cacheThis compilation cache is used during the creation of a personal test area for Debugging purposes. If missing, it shouldn't impact the debugging process.By attempting to debug the espace again, the partial content will be recreated again.
 Release Application FilesThese are the resulting OutSystems application files, mapped on the IIS application server. If they are missing the applications will became offline. There's an automatic internal thread maintaining the content on this folder, so this should never be deleted.One need to redeploy all espaces again to restore the application files. This can be done by restarting the OutSystems Deployment Service on the server, or 1-Click Publishing an all content solution in Service Center, or even use the Re-deploy button for every espace on the Service Center espace details page.
 Personal Test Area FilesThese are the actual application files of the personal test areas for each espace and developer. Deleting them will make the developer lose the ability to run/debug the application by accessing it on the web browser.In order for the developer to restore it's own personal test area for that espace, he needs to perform a run/debug from Service Studio again for that espace.
 Temporary Asp.Net FilesThese are the actual ASP.NET application files loaded in memory by IIS, and if in use, they cannot be deleted until IIS is stopped. Deleting these files will cause a slower first access start to the web application.Automatically generated by IIS upon the first access to the application.


So, based on the tables above, and if you're concerned about disk space usage on your servers to the point of attempting to delete some of the content, here's my tip:

  • Regularly (daily or weekly) clean up the Temporary ASP.NET Files, using the script that's available on the forum topic How To: Optimizing Disk usage by Temporary ASP .NET FilesThis script will eliminate non-used files.
  • In development environment, coordinate with your developers to clean up periodicaly (weekly or monthly) the Personal Test Area Files and eventually, the Partial Compilation Cache
  • Avoid deleting the Full Compilation Cache or the Release Application Files at all on a regular basis. If necessary on one particular occasion, and an all content solution publication is scheduled, you might want to coordinate with the involved teams, but keep in mind that deleting the Release Application Files will cause application downtime, and should be scheduled accordingly.

Let's talk numbers


Finally, let me try to pass on some highlights about some recommended disk sizes.


Estimating the disk space usage of a factory is not easy, since it depends greatly on the number of Software Units, eSpaces and Developers that work on one environment. Additionally, the type of application (and thus the resources included on that application) can really offset any metrics, but  from my experience, and considering the disk fragmentation impact on server performance, I usually recommend the following, based only of the Software Units metric:

  • 36 GB for small factories (< 150 000 Software Units)
  • 73 GB for medium size factories ( < 300 000 Software Units)
  • 146 GB for large size factories (> 300 000 Software Units )

It has been more then enough so far, for both non-production and production environments. Ultimately, a good best practice is to install the Platform Server on a partition other than the system partition, to avoid system drive fragmentation.


Really hope that this information helps you considering the necessary disk space for your environments, and that you got a little more insight on the disk space usage by the Platform Server.


Cheers


Miguel João


Hi Miguel,
Thanks for this Great Guide to disk space usage.
I have another question, can you explain the process of the cleanup of directories (espace_name.01489946960) created in hotdeployment?
When it runs? There are any logs of this operation?
Thank You
Hi Hugo

The automatic cleanup process of the running directory on the Agile Platform, is performed periodically by the OutSystems Deployment Service. The service will attempt to delete any folder within  <platformserver>\running\ that is currently not being used by any IIS virtual directory, and currently by default:
  • On development environments (server mode = Development) it will clear any folder older then 15 minutes
  • On production environments (server mode = Production) it will clear any folder older then 24 hours
However, these parameters weren't always set with these default values, and in development, the value used to be 2 hours. If you have the need to tweak these values, I suggest you get in touch with the OutSystems Technical Support team for solutions on how to change the defaults.

Regarding the period of the cleanup thread, you actually can set it on the Platform Monitoring page of Service Center (under Monitoring), and if you access the Deployment Service details, you'll find the Unused Folders Removal thread, and the time since the last execution.

Regarding the logs, you might find some General Logs from the (system) espace, regarding this cleanup thread if anything was deleted.

Cheers

Miguel Simões João
Hello,

Just some addition information for the people who didn't knew this:

Since our the diskspace become very large and all the above directories where 'cleaned up' a colleague founded out that a windows directory became very large. This directory is: C:\inetpub\logs\LogFiles\W3SVC1. It a directory used for the loggin of IIS. I've found a post on internet where is told what it is for, how you can enable/disable the log files being created and even a script line to clean it up after 30 days it can be found here:

http://social.technet.microsoft.com/Forums/en-US/configmgrgeneral/thread/d989b249-0159-41fc-b78c-1f1d91ce8bb3

Thought it would be usefull regarding this post.

Kind regards,
Evert
Hi Evert

Thanks for the heads up about the IIS logs. That's definately another source of large disk space usage over time.

I would recommend to clear older logs, instead of disabling the logs. Disabling the IIS logs will cause loss of troubleshooting information. So instead, you can create a script to delete logs older then some months, or even compress the old logs, and being similar text, it will have very high compression ratios (>90%).

Cheers

Miguel Simões João
Greetings,

I just found out I had more then a year's worth of IIS log files on all my production front-ends, so I'm reving this topic and adding the script I used to clean them up :)

You can create a Task Scheduler task to run this command regularly:

powershell.exe -ExecutionPolicy ByPass C:\OutsystemsCleanup\Remove-IISLogfiles.ps1 -noprofile – Noninteractive

The powershell script "Remove-IISLogfiles.ps1" is attached to this post, and, in my case, it was placed in the C:\OutsystemsCleanup folder, along with the script to delete unused temporary ASP.Net files. You can, of course, put it anywhere and change the path on the command.

Hello João,

Script is usefull, some addition for people who download it:

Keep in mind that the IIS log file path can be different so they maybe need to change it.
 Keep in mind to adjust number of days to a wishfull number of day (saw that is was 60 days in you script).

Kind regards,
Evert
Hi,

I think it should be a section in the forge for this kind of scripts. They help to save time and deserve to be in the forge.
There is already an idea for that :)

Cheers,
JA