Save the nerves of your computer! Make your programs use less memory without sacrificing their functionality.
This article shows a number of tricks to preserve RAM while your program runs. The code snippets are written in VB6, but the concepts are universal.
Clear old memory before requesting more
Your application might temporarily require 2x the memory it actually uses. Consider the following code snippet:
' Load a picture from a file Pic = LoadPicture("abc.bmp") ' Display the new picture Form1.Picture = Pic
What's wrong here? You may be holding two pictures in the memory at the same time! You are possibly displaying an old picture on Form1 while loading the new one. If the pictures are large, this can slow down your app quite a bit as it suddenly needs to allocate lots of new RAM. If you're running low on RAM, the system may need to access the swap file, which is slow.
The solution is to unload the old picture first. This way you don't need the RAM for that picture any more and the system will probably allocate the freed memory for the new picture. This might cause a flashing effect in the user interface, though, before you get the new picture displayed.
Type your variables properly
Use the smallest data type that covers the use. Using long integers or floating point to store values 0..255 is an overkill. You can sometimes come by with an short integer instead of a string. Some environments (such as .NET) also allow typed enumerations. A byte is usually more than enough to store the values of an Enum.
* = Notes below
Choosing the correct data type is especially important with arrays and user-defined data types. The savings with regular variables are much smaller.
Leave out the formatting
Many ID numbers are formatted series of digits and letters. Examples are SSN, ISBN, ISSN. They are typically shown as something like:
The digits are grouped and separated by hyphens (or other markup). The string ends with a check digit or two, here denoted by
You don't necessarily need to store the ID in the formatted form. You can strip all the separators and the check digit and store the bare digits like this:
When you need to display the ID, add the separators back and calculate the check digit.
You can even convert the bare ID from its string representation to a numeric value. ID numbers typically consist of a limited alphabet, such as digits 0-9 and some letters. If the alphabet consists of, say, 36 characters (0-9, A-Z), you can interpret the ID as a base-36 numeric value. Just convert it to an integer (base-10) and store it in a regular integer variable. Example:
This trick is particularly useful for hexadecimal values (base-16). Each hex digit (0-9A-F) represents a value from 0 to 15. Two hex digits make up a byte (values from 0 to 255). Thus, you can compress hexadecimal numbers by a factor of 2:1. Example: FFFF => 65535, which fits in 2 bytes instead of 4.
Isn't it cumbersome to store a value in a strange datatype and perform manipulations each time you write or store the value? The answer is yes. The solution is to encapsulate the data as a property. Have the property accessors (Get and Let) do the maths and access the underlying variable. Your other code accesses the property, not the raw data.
You can also create your own data types (classes, structures) that are responsible for the encapsulation and storage. You can then reuse these data types throughout your projects and save memory each time. Create your own Date data type, Time type, ISBN type, Price type, Flags type, ... Depending on your programming language, you can provide property accessors, constructors and operators to do the maths for you. This way you can use the data type almost as if it were a built-in one.
Allocate arrays wisely
How do you usually allocate large arrays? Do you regularly allocate "enough" space to cover all imaginable situations? Say you allocate an array of 100,000 records even though the code usually requires just 1000 or 2000 records. That's a problem with statically sized arrays.
The solution is to use dynamic arrays. The following shows various options to do this in VB.
In this method you only allocate the space you need. Before using the array, you calculate how much space you need this time. You allocate the space you need, not the space to cover all possible situations.
' Declare the array and its data type Dim Data() As type ' Estimate the required array size by some method MaxSize = f(...) ' Allocate the space ReDim Data(MaxSize) ... ' After use, deallocate the space Erase Data
You don't necessarily need to get MaxSize exactly right. It's enough if you can estimate an upper bound for it. If you allocate 1000 records and only use 990 of them, it's not a big deal.
Sometimes it is not possible to determine how much to allocate before actually processing the data. In this method you reallocate more space as you need to store more data.
' Declare the array and its data type Dim Data() As type ' Guess an initial size MaxSize = 1000 ' Allocate the initial size ReDim Data(MaxSize) ... Do ' Need to store more data Size = Size + 1 If Size > MaxSize Then ' Entire array consumed, allocate more space ' Come up with a new size MaxSize = MaxSize * 2 ' Reallocate preserving existing data ReDim Preserve Data(MaxSize) End If ' Store the new data Data(Size) = xyz Loop Until... ' After use, deallocate the space Erase Data
With this method it is important to make good guesses for the initial size and the formula you use to grow the size (doubling, tripling etc.). Reallocation is a costly operation, so you should use it sparingly.
Deallocating the extra
Sometimes it happens that you don't need all the space you reserved. In this case you can deallocate the extra while keeping the good data.
' Allocate "enough" space ReDim Data(100000) Do ' Fill up the array logging the actual size used Size = Size + 1 ... Loop Until ... If Size < 80000 Then ' Free the extra bytes if you save "enough" bytes ReDim Preserve Data(Size) End If
Deallocating the extra makes sense if you allocated way too much. As deallocation can be slow, you can skip doing it if it would only free relatively little memory. The way your programming language handles dynamic deallocation is important. It might just free up the extra bytes from the end or it might allocate a new array and copy the data to it, which takes much more time. Test the performance before use.
Clear your globals
Global variables are nasty as it's so easy to forget to clear them. Once your software keeps running never clearing its globals, you end up carrying old data for no use.
Large graphics require lots of space. A 1024x800 picture with full 32bit colors consumes 3 MB. The same picture with 8bit colors (256 colors or 256 gray shades) consumes just 800 kB. You could even get by with just 16 colors or black&white, consuming even less memory (400/100 kB).
To cope with a low memory situation you can downgrade your pictures to less colors. You can even do without graphics altogether.
If you've ever programmed with the (unmanaged) Win32 API, you know it's so easy to forget to release the allocated resources. It can eventually lead to lockups and other problems. Unfortunately, these bugs are difficult to debug.
There are two utilities that can really save your day.
Process files in parts
When processing files, you don't necessarily need to load the entire (big) file in the memory. You can process it line by line, for example, or load only the bytes you need for the task at hand.
Less XML, less memory
XML is a bad format memory-wise. First, XML is always textual data. Expressing numeric values as text takes more space. If you want to express binary data, you need to encode it as text. Second, the verbose start and end tags and whitespace consume even more bytes. You potentially consume more bytes for the markup than for the real data.
XML has its uses as a data exchange format, but don't use it as an internal data format if you want to save memory.©Aivosto Oy -