advertisement

Article:
  Top Ten Data Crunching Tips and Tricks
Subject:   In reply to "read the input into memory" considered harmful
Date:   2005-06-14 08:55:44
From:   jtaylor247
What Greg suggested does not neccessitate a script loading all records into memory first. If you have three functions read(), transform(), and write() you can simply apply these in a read() transform() write() loop that loops once per record leaving only one record in memory at a time.