View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0011624 | MMA | Synchronization | public | 2013-12-18 22:04 | 2014-01-21 19:31 |
| Reporter | rusty | Assigned To | |||
| Priority | immediate | Severity | major | Reproducibility | always |
| Status | closed | Resolution | fixed | ||
| Product Version | 1.0.4 | ||||
| Target Version | 1.0.4 | Fixed in Version | 1.0.4 | ||
| Summary | 0011624: USB Sync: Playlists don't delete from MMA | ||||
| Description | When playlists are unchecked on the auto-sync list, and delete unselected... is enabled, --> playlists aren't deleted after a USB sync: Debug log posted to ftp: 1049 Plug in device Verified that 5 playlists are on auto-sync list -Beatles-not-rated -BeeGees-test-list -Billy Joel-test-list -Good stuff -Videolist2 7361 Clicked auto-sync 2107 Confirmed deletion of Missing metadata.m3u, plf.m3u, test plist 1.m3u 22809 unplugged device --> playlists still appear in MMA!! Generated MMA log W7QUIW613W 22941 Initiated Wi-Fi sync 27195 Accepted MMA confirmation dialog to delete the 3 playlists that should have been deleted previously --> playlists are deleted this time from MMA device Generated MMA Log QC69LQJ1PN | ||||
| Tags | No tags attached. | ||||
| Fixed in build | 214 | ||||
|
|
In MMW log I saw this when deleting the playlists: WMDM: GetStorageFromPath: missing metadata.m3u was not found WMDM: GetStorageFromPath: plf.m3u was not found WMDM: GetStorageFromPath: test plist 1.m3u was not found It means that MMW / Windows did not see the M3U files there when deleting, so it also couldn't remove the M3U files, couldn't it be related to problems described in 0011458 ?? Did you test on KitKat ? Nevertheless MMW uploaded mmstore.db.synced correctly (where the playlist deletions are also reflected.) EDIT: I also cannot reproduce it, tested MMA 203, MMW 1681, Galaxy Nexus, JB 4.2 |
|
|
This was tested on a GS3 running JB 4.1. I thought it might be related to 0011458 due to the changes in how devices recognize files on the device. |
|
|
Fixed in build 205 |
|
|
Tested 205 as follows: In MMW, deselected the following Playlists from the auto-sync list: Aimee-top-rated Jogging ...and selected the following Playlists to the auto-sync list: missing metadata mmalist1 1 1905 Initiated sync to GS3/JB 4.1.2 2 6095 Disconected the device (note: MMW debug log is posted to ftp) Ran MMA --> scan starts --> debug logs automatically uploaded after 0000010:0000045 seconds (1:57am) Manually generated MMA log EOMWLGTRB8 Upon examining playlists: Newly added playlists were added to MMA, BUT Deselected playlists remain in MMA (and in Play Music as well)! |
|
|
Fixed in build 206 It was different issue that was caused by existing playlist in MMA with same path but different GUID. |
|
|
The playlists are still not deleted in build 206 (tested on GS3/JB 4.1.2). Debug log was automatically generated during the scan process, and then a couple of minutes later I manually generated debug log ID: KTYE6FRFVX EDIT: Note also, that if I do a second USB sync after performing the above test, MMW again prompts to delete the playlists in question, which implies that either: a) they weren't deleted in the first place b) MMA/MediaStore interactions recreated the .m3u playlist files There's also a new related issue at 0011646 in which debug logs are generated even when the only action is to add a playlist to the sync list. EDIT2: I did some more testing and found that if I do the USB Sync and then run Player Pro on the Android device, that the playlists are not deleted in Player Pro either--what happens is that the _contents_ of the playlist are deleted, but the playlist itself remains. This makes me wonder whether perhaps MMW may have a part in this bug. |
|
|
Re: EDIT2: There is no new MMW log so I suppose that the reason is still the same (0011624:0038838), i.e. MTP refresh issue (Windows/MMW did not see the M3U files so it also couldn't remove them) |
|
|
Re. Edit2: I can see in Android that the playlists in the /Playlists directory have been deleted as expected by MMW during USB Sync. However, for whatever reason, the Android mediastore doesn't seem to reflect the changes. Similarly, if I delete /Playlists/Playlist.m3u from the Android device using Windows Explorer, and then run PlayerPro and refresh the mediastore, Playlist.m3u remains in the mediastore. So I don't know if this is a problem with Androids MTP implementation OR my assumption that deleting the .m3u files in the /Playlists directory should result in a change to the playlists in the mediastore (i.e. should playlists be deleted in some other fashion??). |
|
|
I've posted new logs generated with MMW 1683 / MMA 210: MMW log showing sync in which: Playlist furr was added to auto-sync list Playlist missing metadata was removed from auto-sync list Playlist mmalist1 was removed from auto-sync list ...is uploaded to ftp After the sync, a debug log was automatically generated by MMA at 10am. I then manually generated the following debug: SIDOCIHPNA |
|
|
Fixed in build 211 |
|
|
Tested build 211 and the debug logs are no longer generated however, when unchecking Playlist 'Furr', and adding 'missing metadata' and 'mmalist1' to the auto-sync list and then doing a USB sync -->The two playlists are added to MMA but 'Furr' isn't deleted, despite the fact that Furr.m3u is removed from the extSDCard/Playlists directory. Debug log ID: PFVU0KUP4Z |
|
|
Assigning to Ludek because it looks like issue on MMW side. There are no records about deletion in sync_operations table. When I just unselect some playlist without syncing anything else, the sync_operations table is not even created. But I have fixed freezing of USB sync in case of this failure. (build 212) |
|
|
sync_operations table is always created within mmstore.db.synced, I don't see how it couldn't If it is really so that M3U is deleted, but there is no record about the playlist deletion in the sync_operations table then the only reason I could see is that the M3U path in the MMA's DB differs (the playlist is stored under different M3U path in MMA DB), because MMW deletes playlists from MMA DB based on M3U path. But watching at log (MMA/bugs/bug11624/MMW_bug_11624.log) the furr.m3u really wasn't deleted by MMW, I see only these items were deleted: /storage/extSdCard/Music/Elana Jagoda/Zum Gali Gali/01 01 Zum Gali Gali.mp3 /storage/extSdCard/Music/Jock Jams/Chicken # three/00 The Chicken Dan.mp3 /storage/extSdCard/Music/Unknown/Alas I Cannot Swim/01 Ghosts.mp3 /storage/extSdCard/Playlists/missing metadata.m3u /storage/extSdCard/Playlists/mmalist1.m3u Actually I see that the \Playlists\furr.m3u was created/synced by MMW, but this is most probably the old log from the previous sync (0011624:0039010), right? Nevertheless I haven't been able to replicate this issue in any way and also haven't found a reason for it by code revision on MMW side. |
|
|
To hopefully clarify what's happening, I reran this test using MMA build 211 / MMW 1683 as follows (MMW debug log numbers are indicated--the log is posted to the ftp server): 1 run mm 2 934 Initiate wifi sync 3 55600 plugged in device 55600, and unchecked Playlists 'missind metadata' , 'mmalist1' and checked 'Furr' 4 62660 initiated usb sync (having confirmed that MMA process wasn't running) 5 78399 unplugged device 6 Ran mma and waited for scan to complete 7 Clicked Playlists node -->Playlists 'missing metadata', 'mmalist1' have not been deleted! Generated debug MMA log J6JHYGQJ3G I proceeded with the following just in case the logs would yield some useful data re. why wi-fi sync works but usb sync doesn't. 8 78591 Initiated WiFi Sync 9 82872 I accept MMA Prompt for deletion of 'missing metadata', 'mmalist1' 10 82952 Sync completes, and I click the Playlists node --> Playlist node displays correctly i.e. 'missing metadata', 'mmalist1' no longer appear. Generated MMA debug log GLREB0BYEM |
|
|
I don't see a problem in the MMW log, MMW deleted /storage/extSdCard/Playlists/missing metadata.m3u /storage/extSdCard/Playlists/mmalist1.m3u both from MMA's DB and also physically on the device. Rusty, could you please attach your MMA DB (\MediaMonkey\files\mmstore.db) so that I could verify that the absolute M3U path doesn't differ for whatever reason (e.g. storage prefix)? It would be also valuable if you could get \MediaMonkey\files\mmstore.db.synced that is created by MMW for MMA between steps 4 & 5 (to confirm that everything is written correctly there). BTW: I re-tested this again with MMA 211 and MMW 1684 using these new steps you provided and I was not able to replicate. |
|
|
I've posted the /MediaMonkey and /Playlists directory for both the internal memory and SDCard at: - step 0 (after getting the device into the proper configuration using wi-fi sync) - step 4.5 (after step 4 and before step 5) - step 7.5 (after step 7--just in case MMA's scanning operation is somehow causing the bug) |
|
|
I can confirm that the playlists really were not deleted from mmstore.db.synced (sdcard.11624.step4.5.zip). What is even stranger that performing this SQL: DELETE FROM playlists WHERE _data = '/storage/extSdCard/Playlists/missing metadata.m3u' does nothing on that DB although it should delete the playlist, because the playlist exists in the DB with this _data value! Out of curiosity executing the same SQL works perfectly for the Good Stuff playlist: DELETE FROM playlists WHERE _data = '/storage/extSdCard/Playlists/Good stuff.m3u' The same happens for 'mmalist1.m3u' versus 'videolist2.m3u', mmalist1.m3u fails to delete although videolist2.m3u is deleted, this is really strange (tested with SQLiteStudio), studing more... |
|
|
I haven't found the real cause of this curious problem, but I found that although SELECT * FROM playlists WHERE _data = '/storage/extSdCard/Playlists/mmalist1.m3u' doesn't work, using SELECT * FROM playlists WHERE _data LIKE '/storage/extSdCard/Playlists/mmalist1.m3u' works for unknown reason (although both should return the same), so as a quick workaround I could use 'LIKE' operator instead of '=' when deleting, but this doesn't explain the real cause and in addition LIKE tends to be slower when index is used (not this case though). |
|
|
After further playing with the Rusty's DB in SQLiteStudio I found another and better workaround for this curious SQLite issue. Once I created index on the playlists._data column, the '=' operator started to work on all playlists rows! So Marek please create index on playlists._data, it will have also performance advantages (when querying). And please create also index on playlists.guid and name columns. EDIT: Assigned back to me, the added index did not help, it was test a test error (I used LIKE instead of =) Nevertheless it seems that the real reason is failed integrity check that is presented even in the original mmstore.db (sdcard.11624.step0.zip), it fails exactly for rows 6 and 7 of the playlists table (rowid 6,7 missing from index sqlite_autoindex_playlists_1), note that rows 6&7 are the rows for which this strange issue happen. For the integrity check I opened the mmstore.db in SQLiteStudio, right-clicked it -> Integrity Check. This means that the database is corrupted. @Rusty, is the problem reproducable if you uninstall MMA, delete mmstore.db and re-install (i.e. fresh install)? How did the affacted playlists (mmalist1, missing metadata) got originally to the DB? Via WiFi or USB sync? EDIT: Actually before uninstall try to reboot the phone, re-connect and attach the mmstore.db again. I remember from the past that MTP refresh issue could cause DB corruption, it did not refresh new size of the file over MTP and the file was downloaded incomplete, so this will show us whether it could be the issue. But it is less probable, because in your log I saw that MMW successfully downloaded mmstore.db.copy that MMA creates for MMW based on MMW request file. |
|
|
I think that it is really possible that issue is that the file is not well refreshed over MTP. As you know I tried to implement new way of refreshing of MediaStore. It wasn't working quite well on Nexus 7 with KitKat (and other devices). It was working for new added files but it wasn't working well for deletions of files and updates (overwrites) of files. But I figure out that refreshing of playlist files works fine. They were processed slightly differently. So I tried to use this functionality for control files of USB sync. Database files are now not directly overwritten when updated but deleted, MediaStore is refreshed, new file is copied and refreshed again. And it looks like it works. Even with Nexus 7 with Kitkat. Please try it. Fixed in build 213 |
|
|
Sorry for the late feedback. The bug occurs even when i delete the DB. The DB is recreated as follows: 1 from apps delete MMA settings 2 delete /MediaMonkey directories 3 run MMA and wait 15 minutes for it to scan content 4 wifi sync with MMW (during which it performs matching operations 5 perform the set of steps described at 0011624:0039037 to trigger the bug |
|
|
I tested build 213 on a clean install of MMA exactly as described at 0011624:0039064 A new set of intermediate DBs have been uploaded to the ftp server. |
|
|
Integrity check again fails on rows 5,6 of the playlists table, i.e. for 'mmalist' and 'missing metadata list' again, and it exhibits the same problem as before. This makes me think that the DB was not corrupted by MTP transfer, but it could be probably corrupted also in the original MMA DB somehow by writing these playlists to the database during WiFi sync? Marek, is there a way for Rusty to get the internal database (on internal root device memory) so that we could confirm my speculation? The really strange thing is that only those two playlists are affected, but because they are the last two in the table then I suppose there can be a hidden exception before writing the last two playlists to MMA DB during WiFI sync? Just speculations. As I wrote above, as a workaround I could use LIKE instead of = operator, but this doesn't explain the real cause of DB inconsistence. |
|
|
No it is not possible to get the DB without root. Rusty, you have made a completely clean installation. I.e. you have deleted the MediaMonkey directory too, right ? I can look at logs to see whether there is anything wrong. But it is really strange...again these two playlists. Rusty could you please generate MMA logs for me ? |
|
|
Marek, yes I made a completely clean install. Generated new logs + db snapshots for deletion of playlist 'furr'. Log ID: BWVF6YAU5R |
|
|
Note that I can also reproduce the issue now, integrity check fails on the last row of the playlists table for me (the 6th in my case). I have rooted device and when I transferred the mmstore.db directly from root (original location: root/data/data/com.ventismedia.android.mediamonkeybeta/databases/) then the integrity check fails same way in SQLiteStudio. But if I execute 'PRAGMA integrity_check;' in the SQLite Debugger android app then it wrote "OK" for both DBs (orig. and on SD card) and also it doesn't exhibit the original problem in the SQLite Debugger android app !!! This is strange and indicates that there is probably an incompatibility in handing SQLite databases in Android apps (SQLite Debugger, MMA) versus PC apps (SQLiteStudio, MMW) or there is an MTP transfer problem (less probable as I transferred the files several times also after phone reboot). EDIT: It is not MTP transfer issue, because if I copied the DB back from PC to the device then it doesn't exhibit the problem in SQLite Debugger Android app, but it exhibits the problem in SQLiteSudio PC app so unfortunatelly it is really an incompatibility between Android's SQLite and PC's SQLite. Going to try more SQLite PC alternatives to confirm my speculations. But it is hard to find some, because e.g. 'SQLite Man' and 'SQLite Browser' can't open the DB at all, 'SQLite Expert', 'SQLite Manager' and 'SQLite Spy' opens the DB, but queries fail on collation UNICODE. BTW: I found another workaround beside using the LIKE operator, executing "REINDEX playlists" also workarounds this issue (which seems cleaner workaround). |
|
|
Further testing shown that executing REINDEX on PC (SQLiteStudio 2.1.5) fixes the issue, but subseqent re-copying to the phone causes the exaclty same issue on the phone (tested Android's SQLite Debugger) until REINDEX is executed on Android. i.e. There is an incompatibility in indexation between PC and Android. I wonder whether it could be related to UNICODE collation? Maybe it is handled differently? |
|
|
As discussed with Marek over Skype, the solution for now is to call REINDEX on both sides when opening DB. I tried execute REINDEX on my MMW DB (20k tracks) and it took only 1.6 seconds, so it sems to be usable. i.e. 1) MMW will call REINDEX when opening mmstore.db 2) MMA will call REINDEX when opening mmstore.db.synced |
|
|
I fixed 1) on MMW side and confirmed that it solves this strange problem, REINDEX result was "0 row(s) affected in 0.016128 second(s)" and then integrity check was "OK" and the problematic playlist was deleted in MMA as expected. Fixed in build 1685. Assigned 2) to Marek. EDIT: It really seems to be caused by the UNICODE collation, because 'REINDEX UNICODE' also solves the problem, it looks like on Android the UNICODE collation is somehow limited to first several chars only, because it affects only strings with same longer prefixes (e.g. the m3u paths) |
|
|
Fixed in build 214 |
|
|
Verified 220. |
