View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0004249 | MMW v4 | Playlist / Search | public | 2008-01-04 14:56 | 2008-01-05 00:20 |
| Reporter | rusty | Assigned To | |||
| Priority | immediate | Severity | major | Reproducibility | always |
| Status | resolved | Resolution | fixed | ||
| Product Version | 3.0 | ||||
| Fixed in Version | 3.0 | ||||
| Summary | 0004249: Search is slow on large DBs | ||||
| Description | As reported by a user, searches of large databases are slower than they should be. This is problematic if we are going to be advertising MM as the solution for large 100k+ music collections. The problems described by the user are: Library (database): I added my 0000047:0000080,000 item collection on a fast machine. When using the search function I have four issues I wasn't able to find answers for in the forums that I'd like your help with: 1) It takes several seconds for results to appear regardless whether it matches 10 or 10,000 items. I desire 100ms search time (real-time), or at least as fast as iTunes (never more than 2 seconds). Currently neither is true. Many searches return 1 or some small number of items within 1 second, while the bulk takes an additional 5 seconds on average (sometimes more, up to about 10 seconds). What is displayed early or late seems to depend on it's location in the database (i.e. Zoo takes a while before displaying anything, whereas Abba prints some immediately). 2) There is no search progress indicator? Sometimes as the search is incrementally expanding it says 'Reading tracks' (or similar) at the bottom of the screen, but it seems this has more to do with reading tag information than the search progress. How do I know when the search is complete? 3) I suspect the SQLite database is not fully cached. Is there a way to influence this, i.e. set my own SQLlite cache size: 'PRAGMA cache_size = 100000'. The database is ~90MB, and I'm happy to provide even 500MB of RAM for real-time performance (this machine has 4GB RAM). 4) Is it possible to constrain the search to improve performance? What does the select string look like now? It's my desire to only search artist, album, and title, which I know requires only a few tens of milliseconds on this machine to search all 80,000 items and return title,artist,album,length,track (which are all that I have displayed). I have a secondary SQLite database I created myself that contains 0000025:0000030 columns of data for every mp3. For example, this takes tens of milliseconds: select TITLE,ARTIST,ALBUM,TIME,TRACKNUM from mp3 where ARTIST like "%pantera%" OR ALBUM like "%pantera%" or TITLE like "%pantera%"; In MediaMonkey I have only Title, Artist, Album, Length, and Track # displayed (just like the select statement above), and when I search for "pantera" it takes ~8 seconds to complete. ~30ms vs. ~8 seconds is not justifiable. I'd be more than happy to test out an alpha/developer builds of MM and work with you to improve performance. I'm sure the entire MM community would appreciate this effort. | ||||
| Tags | No tags attached. | ||||
| Fixed in build | 1128 | ||||
|
|
I asked the user for more info (e.g. directly his DB) in order to find out more. So far it seems that we can't make any significant speed up without implementing full-text search. I'm not sure about his observations in 4), Ludek made similar tests and got only some 5x better performance, not that big difference. We'll see when I get more info from the user... Btw, his idea 3) is quite interesting, it could make MM faster for users who have a lot of unused RAM. I asked him to test it (using a script), if it works well, we could implement something like this in some future version. Re. full-text search - there is currently something new being implemented in SQLite, I'll give it a try, if it works well, it could get into MM 3.1 or 4.0 and make searches really fast then. Until then, I'm afraid that we can't do much. |
|
|
A 5x improvement is still fairly significant--i would suggest that if this is found to be true, that we add a config entry to the .ini file that would allow searches in the search toolbar to be restricted to Artist, Album, Title (it should have no effect in the search dialog). |
|
|
Fixed in build 1128. Addition of: [Options] QuickDBSearch=1 to MM.ini file turns this mode on (which means that the only searched fields are Title, Album, Artist, Genre, Album Artist, Composer, Conductor and Lyricist). According to my measurements with 107k tracks in DB, it's about 10x faster than the previous version. Note that I also found a way how to increase speed of the standard (i.e. full) search, it's about 2x faster now! |
