OSCommerce Product Manager

OSCommerce Product Manager for Windows
Tasklist

FS#83 - Higher limits for big stores.

Attached to Project: OSCommerce Product Manager
Opened by Mario A. Valdez-Ramirez (mvaldez) - Tuesday, 24 August 2004, 06:23 GMT-6
Last edited by Mario A. Valdez-Ramirez (mvaldez) - Tuesday, 04 April 2006, 06:16 GMT-6
Task Type Bug Report
Category Backend / Core
Status Closed
Assigned To Mario A. Valdez-Ramirez (mvaldez)
Operating System All
Severity Low
Priority Urgent
Reported Version any
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

One user reported in the forums that the application is producing Range Check errors with a big big big store.

The size of the store is reported to be:
23000 products in 1500 categories.

The OSCPMWin was not designed to handle such large store. We need to review the code for 3 things:

1) To increase the limits safely (check for memory consumption on top usage).

2) Add range checking to all operations importing data from the database. The error the user received was an exception handled directly by Delphi, the state of the data in the memory structures is unknown. The application should stop, warn the user and end any further processing.

3) Check for a way to handle those limits dinamically. Unless reviewing point 1 results in that we can move the limits up without any danger, we should look for another way to allocate memory on-demand.

Severity as "Major", because the application cannot be used.
Priority as "Medium-High", just to be on the safe side.
This task depends upon

Closed by  Mario A. Valdez-Ramirez (mvaldez)
Tuesday, 04 April 2006, 06:16 GMT-6
Reason for closing:  
Comment by Mario A. Valdez-Ramirez (mvaldez) - Wednesday, 25 August 2004, 02:08 GMT-6

Just for the record, current limits are as follow:

for product information listing
opmC_Max_Products_PerList = 8192;
opmC_Max_Descriptions_PerList = 40960;
opmC_Max_Languages_PerList = 16;
opmC_Max_Categories_PerList = 512;
opmC_Max_Menufacturers_PerList = 512;
opmC_Max_Taxes_PerList = 64;

for image handling
opmC_Min_ImgX = 20;
opmC_Max_ImgX = 1024;
opmC_Min_ImgY = 20;
opmC_Max_ImgY = 1024;

for main window size
opmC_MinWindow_XSize = 300;
opmC_MaxWindow_XSize = 2000;
opmC_MinWindow_YSize = 200;
opmC_MaxWindow_YSize = 2000;

for string lenghts
opmC_DBUsername_MaxLen = 128;
opmC_DBPassword_MaxLen = 128;
opmC_SSHUsername_MaxLen = 128;
opmC_SSHPassword_MaxLen = 128;
opmC_ProdModel_MaxLen = 12;
opmC_ProdName_MaxLen = 64;
opmC_ProdWeight_MaxLen = 7;
opmC_ProdPrice_MaxLen = 17;
opmC_ProdQuantity_MaxLen = 16;
opmC_ProdURL_MaxLen = 255;
opmC_ProdImageURL_MaxLen = 64;
opmC_ProdDescription_MaxLen = 1024;
opmC_ProdSpecial_MaxLen = 12;
opmC_CatName_MaxLen = 32;
opmC_LangName_MaxLen = 32;
opmC_ManName_MaxLen = 32;

as for remembered config data
opmC_MaxPast_Servers = 10;
opmC_MaxPast_Usernames = 10;
opmC_MaxPast_SSHUsernames = 10;
opmC_MaxPast_Databases = 10;
opmC_MaxPast_ImgURL = 10;
opmC_MaxPast_UploadURL = 10;
opmC_MaxPast_Proxy = 10;


These values are valid as of OSCPMWin 0.1.0.333.

Comment by Mario A. Valdez-Ramirez (mvaldez) - Friday, 27 August 2004, 04:55 GMT-6

This is probably wrong:
opmC_Max_Descriptions_PerList = 40960;


It should be:
opmC_Max_Descriptions_PerList = opmC_Max_Products_PerList * opmC_Max_Languages_PerList

= 8192 * 16
= 131072

Oops!
Comment by Mario A. Valdez-Ramirez (mvaldez) - Friday, 27 August 2004, 04:57 GMT-6

Limits are (from 0.1.0.346):

Max products from 8192 to 32768.
Max descriptions from 40960 to 524288.
Max categories from 512 to 2048.
Max manufacturers from 512 to 2048.
Max taxes from 64 to 128

Closing as fixed.

At least until somebody else ask us about a super big store... yuks.

Later we can reopen this bug to check and improve memory usage.
Comment by Mario A. Valdez-Ramirez (mvaldez) - Wednesday, 01 June 2005, 13:52 GMT-6


It happened... somebody has a bigger store.

One user reported using more than 2500 categories.

We will increase again the limits. I suggest double them.


On a related problem, the application is raising an exception, not a friendly, recoverable error message. I suggest to check always the record count so the code doesn't go further than the array limits. Remember, no input data can be trusted unless is in full control of the application.

Reopened. Let's double the base limit values.
Comment by Mario A. Valdez-Ramirez (mvaldez) - Wednesday, 29 June 2005, 20:55 GMT-6
New limits are:

Max products from 32768 to 35000.
Max descriptions from 700000 to 1000000.
Max languages from 16 to 20.
Max categories from 2048 to 5000.
Max manufacturers from 2048 to 5000.
Max taxes from 128 to 150.


Fixed.
Closed.

Comment by Mario A. Valdez-Ramirez (mvaldez) - Wednesday, 29 June 2005, 20:57 GMT-6
Error, Max descriptions was changed from 524288 to 700000.
Comment by Mario A. Valdez-Ramirez (mvaldez) - Wednesday, 29 March 2006, 10:34 GMT-6

Reopening.

Just when I think the limits are high enough, somebody comes and shows me I'm wrong.

We really should use dynamic-sized arrays.
Comment by Mario A. Valdez-Ramirez (mvaldez) - Tuesday, 04 April 2006, 06:16 GMT-6


Modified oscpmdata.pas to increase the limits of the application. Max products from 35000 to 50000. Max descriptions from 700000 to 1000000. Max categories from 5000 to 10000. Max manufacturers from 5000 to 8000. This fixes Bug #83 (which was reopened).


Closing as fixed... until another user request more memory. :)

Loading...