OSCommerce Product Manager

OSCommerce Product Manager for Windows
Tasklist

FS#317 - HTTP-query data should be compressed.

Attached to Project: OSCommerce Product Manager
Opened by Mario A. Valdez-Ramirez (mvaldez) - Saturday, 25 March 2006, 20:20 GMT
Last edited by Mario A. Valdez-Ramirez (mvaldez) - Sunday, 02 April 2006, 21:54 GMT
Task Type Bug Report
Category Backend / Core
Status Closed
Assigned To Mario A. Valdez-Ramirez (mvaldez)
Operating System All
Severity Low
Priority Low
Reported Version any
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Data received over HTTP (SQL query results) should be compressed.

We have tested Deflate algorithm for compressing individual fields of data (not the whole stream), getting compression rates of around 35% (the resulting data is around one third of the original).
This task depends upon

Closed by  Mario A. Valdez-Ramirez (mvaldez)
Sunday, 02 April 2006, 21:54 GMT
Reason for closing:  
Comment by Mario A. Valdez-Ramirez (mvaldez) - Saturday, 25 March 2006, 23:21 GMT
On testing, we are getting 20% to 25% (4-5 to 1 ratios) of compression using deflate, with level 1, on the whole recordset from the HTTP-SQL query.

Comment by Mario A. Valdez-Ramirez (mvaldez) - Monday, 27 March 2006, 08:11 GMT
Changing to zlib compression (no deflate) to be able to use the decompression functions from the Indy library.

We have tested the ZlibEx unit (a direct OBJ library) which is faster than Indy and it is released under a MIT-style license. However, the differences were not that important.

Using the sampling profiler we get a 4% CPU time with zlibex, and a 8% CPU time with Indy (Mixing small and large compressed data blocks).

Using zlibex will increase the complexity if maintaining the source code. Besides, zlibex only work with P6-level processors (from Pentium II).

Comment by Mario A. Valdez-Ramirez (mvaldez) - Monday, 27 March 2006, 08:15 GMT

Using only small data loads (the most common usage) we get a 0.76% with Indy and 0.56% with ZLibEx. Again, it is not that important.

We will not use ZlibEx (at least for now).

Comment by Mario A. Valdez-Ramirez (mvaldez) - Monday, 27 March 2006, 08:16 GMT

I would like to use something like LZO (http://www.oberhumer.com/opensource/lzo/) which have support in the Delphi side but it lacks support in the PHP side.
Comment by Mario A. Valdez-Ramirez (mvaldez) - Monday, 27 March 2006, 08:21 GMT

Forgot to mention:

The sampling profiler for Delphi is developed by Eric Grange (from GLScene.org), and it is located at: http://glscene.sourceforge.net/misc/SamplingProfiler.zip
Comment by Mario A. Valdez-Ramirez (mvaldez) - Monday, 27 March 2006, 08:31 GMT

This is partially done. I am planning to convert all remote functions to return recordsets just like the HTTP-SQL queries. In that way, we would use less code and would use an uniform way to request remote data to the server-side script.
Comment by Mario A. Valdez-Ramirez (mvaldez) - Monday, 27 March 2006, 08:37 GMT

Closing this.
Comment by Mario A. Valdez-Ramirez (mvaldez) - Sunday, 02 April 2006, 21:54 GMT

This bug to be reopened if someday we get LZO support in the PHP side.

Loading...