This is the first of what will most likely be many postings related to data management and Flex. Our opening topic is a brief discussion on large data sets and how to efficiently transfer them from your application server to the client. I’m going to assume you are familiar with MXML and the components and services that are a part of the Flex runtime.
This post will refer to many examples that are found here. Instructions for installing the samples can be found in a README included in the zip.
Our problem is simple. We need to display a lot of data, and the user does not want to wait too long for it to appear. The more data a user requests, the longer it takes for the data to display. What we need to do is a find a way to download data in pieces so that the user can view a smaller amount of data immediately and then only wait a brief amount of time as they request additional data. To keep the problem domain simple we are only going to discuss improving the actual retrieval of data. We’ll leave discussions on sorting, modification, filtering, etc. for another time.
Example 1 can help you visualize this problem. The data is census information that I got from The UCI KDD Archive and reduced to 20,000 records. Start by downloading a few records, perhaps 10. This doesn’t take long at all. If we increase that number to 100 the response is usually a little slower (these numbers can vary depending on your system setup, for example whether the database and application servers are on the same machine in addition to where you run the Flex app). Increase the number to 1000 and you’ll see a bigger jump. Finally 5000 and the time is becoming painful. Painful enough that an end-user might give up.