Tag Archive for 'programming'

String Concatenation (Memory Buffer)

Published
by
rs
on July 20, 2006
in Technical
. 0 Comments

While working with a certain web application, I just found my self trapped on a string concatination problem. What I’m tryin’ to do is concatenate 2 strings and assigning the output to the first string. My code goes something like this:

PSEUDO CODE

var myString=”";

for(i=1;myQuery.recordCount;i++){
     myString = myString & myQuery.record[i];
}

note:
Let say myQuery is a SQL query which returns
all the fields and its value from a certain table

Looks simple huh? but the problem is, as the record count increases the performance decreases.
say myQuery returns 20 columns and 60,000 rows, using the code above will makes you wait forever. This kind of problem had never been totally solved until the .Net stringbuilder exist.
Since I am not using .Net, I have to find my own way to fix this problem.

What happen behind the code is that, the code creates a new myString object containing the result of the concatenation and discards the old myString. The value of myString is now then temporarily saved in a single memory buffer. So if the loop is repeated several times, the value of myString goes larger and for every 1 loop, the code retrieves that large data from a memory and then saves it again and again and again. And as the data inside the myString goes larger, the performance slows down.

I came with this idea wherein instead of saving the data in a single buffer, why not divide the data and saves it into different memory buffer, in that case, the size of the data in a single buffer will never be too large for it to slow down the performance of the system.

here’s the new code:


PSEUDO CODE

var myString=”";
var x=0;
var j=1;
var myString[1]=”",….myString[20];

for(i=1;i=20;i++){
     x=myQuery.recordCount*i/20;
     for(j;myQuery.recordCount;j++){
          myString[i]=myString[i] & myQuery.record[j];
          if(j==x){
               break;
          }
     }
}

myString = myString[1] & myStrinf[2]……..myString[20]

This code will divide the data into 20 parts and assign each data to 20 different variables.
Once the looping is done, all the data will be set into a single variable called myString.

Using these codes, it changed my waiting time from 15mins to less than a minute.