View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0011799 | MMW v4 | Other | public | 2014-01-30 04:33 | 2020-06-16 09:09 |
| Reporter | peke | Assigned To | |||
| Priority | urgent | Severity | crash | Reproducibility | have not tried |
| Status | closed | Resolution | fixed | ||
| Product Version | 4.1 | ||||
| Target Version | 4.1.1 | Fixed in Version | 4.1.1 | ||
| Summary | 0011799: Scanning: Full filename and path larger than 260 chars can result in MMW crash | ||||
| Description | 1. For some users using MMW 4.1.0.1690 on Windows 8.1 and scanning Paths that contain files resulting in full path+filename larger than 260 chars result in MMW crash. Scanning same path and files on MMW 4.0.7.1511 there is no crash, but MMW ignores those files and do not add them to library. Note: I'm not being able to replicate crash on Windows 7 2. Double click in Windows explorer do not work and show dialog that path/drive is not accessible. 3. Drag and Drop file do not work when File is dragged from Windows Explorer. NOTE: AIMP player do not have any issue 4. MMW can't handle shorten path+filename as served by Windows C:\Users\Peke\Desktop\MEDIAM~1\Music\ORGANI~1\AUDIOB~1\TESTIN~1\ARTHUR~1.CLA\LADYIN~1.1\Linda Ronstadt & Aaron Neville\12. Linda Ronstadt & Aaron Neville - Don't Know Much\12. Linda Ronstadt & Aaron Neville - Don't Know Much.flac 5. MMW do not add such files in Library at all (Related to 1.) 6. My computer Tree node do not show any files in the folder: "C:\Users\Peke\Desktop\MediaMonkey bug testing\Music\organized music into folders to test long filenames\Audiobooks\Testing folder for bugs and regressions\Arthur C. Clarke & Gentry Lee - All 6 Rama Series Audio Books\Lady in Red Vol. 1\Linda Ronstadt & Aaron Neville\12. Linda Ronstadt & Aaron Neville - Don't Know Much\" NOTE: Manually entering file into File -> Open URL or File.... plays the file Sample file used in testing is: "C:\Users\Peke\Desktop\MediaMonkey bug testing\Music\organized music into folders to test long filenames\Audiobooks\Testing folder for bugs and regressions\Arthur C. Clarke & Gentry Lee - All 6 Rama Series Audio Books\Lady in Red Vol. 1\Linda Ronstadt & Aaron Neville\12. Linda Ronstadt & Aaron Neville - Don't Know Much\12. Linda Ronstadt & Aaron Neville - Don't Know Much.flac" Total Commander is used to created that long Path. LOG Files are uploaded to FTP. | ||||
| Additional Information | TEP-335898 FZM-362109 http://www.mediamonkey.com/forum/viewtopic.php?f=7&t=75324 http://www.mediamonkey.com/forum/viewtopic.php?f=13&t=75577 KCH-355360 XQB-912252 (with debug log) http://www.mediamonkey.com/forum/viewtopic.php?p=385115#p385115 JLV-445292 | ||||
| Tags | No tags attached. | ||||
| Fixed in build | 1695 | ||||
| related to | 0010626 | closed | Ludek | MMW v4 | MM sometimes creates files in 8.3 filename form (Windows 8) |
| related to | 0010280 | closed | petr | MMW v4 | After file edits MM adds duplicate database entries with 8.3 filenames |
| related to | 0010692 | closed | Ludek | MMW v4 | MM 4.1 creates 8.3 filenames while tagging when running in VM on Max OS (regression in 4.1) |
| related to | 0011900 | resolved | Ludek | MMW v4 | Format: Better handling of Corrupted and Incorrect files |
| related to | 0011944 | new | Ludek | MMW v4 | File Open: should be made to act according to double click |
| related to | 0012459 | closed | Ludek | MMW v4 | Locate Moved/Missing: Full filename and path larger than 260 chars can result in MMW crash |
| related to | 0014880 | closed | michal | MMW 5 | Sync Tags: Sync tags fail in some cases |
| child of | 0008020 | closed | michal | MMW v4 | f_video: MM is not importing/playing some MOV Files |
| child of | 0012488 | closed | Ludek | MMW v4 | LongFilenames: NFS/NTFS/SMB 3.0 Long Filenames support up to 32k path |
| child of | 0012868 | closed | michal | MMW 5 | Ability to support long filepaths (> 260 chars) on Windows |
|
|
1) I tested on Windows 8.1 and Win 8.1 doesn't allow me to create such a long path at all (unlike Win7). So to test I created that long path on external memory stick and tested on Win 8.1: - Total Commander did not allow me to open the full path - Windows explorer could browse whole that path - MM skipped the file during scan (no crash), shown the folder without files under my computer node (i.e. symptoms of 5,6) In the ticket TEP-335898 I saw that user scanned 'V:' so probably also external HDD where music was copied from MAC (based on the path info). But even if I created the same 266 chars long FLAC path as the file on which it crashed for the user, I could not reproduce the crash during scan, the file was scanned OK on Win 8.1, so I guess that it was this particular FLAC (file tag content) that caused the crash: V:\Music\Mac Sync\Instrumental\Lossless\Greenwood, Jonny and Penderecki, Krzysztof\Threnody for the Victims of Hiroshima - Popcorn Superhet Receiver - Polymorphia - 48 Responses to Polymorphia\12. Jonny Greenwood - 48 Responses to Polymorphia_ Three Oak Leaves.flac @Peke, could you get this particular flac file from the user to confirm my speculation? EDIT: I asked the user for the file via the ticket 2,3,4) seems to be related to changes in 0010626 and 0010280 (fixed in 1691) 5,6) was always the case also in 4.0.7, strange that it works for 266 chars long path, but doesn't work for 377 chars long path because system's function FindFirstFile doesn't find it, it seems that only workaround is to put the short filename version (with wildchars) as the first parameter of FindFirstFile, but it has some limitations too, see: http://msdn.microsoft.com/en-us/library/windows/desktop/aa364418(v=vs.85).aspx |
|
|
1) Moving same file out of that folder into V:\temp\ scan it without problem. V: is FlexRAIDFS Which is possible cause of the problem with large filenames. 1a) It is clearly that using "\\?\" prefix allow up to 32k full path. http://msdn.microsoft.com/en-us/library/aa365247%28VS.85%29.aspx http://superuser.com/questions/320801/alternative-to-windows-explorer-for-long-path-names http://www.codinghorror.com/blog/2006/11/filesystem-paths-how-long-is-too-long.html 1b) Most flash drives uses extFAT are you sure that this is not the case. |
|
|
2,3,4 Verified 1691 |
|
|
5,6 I get similar behavior on UNC files with paths >260 chars |
|
|
Because of the number of possible issues with long paths on Windows, the only solution is to ignore/skip filenames > 260 chars during scan. Such a files should not match import criteria otherwise we can always have issues on various file systems / system configs. |
|
|
Shouldn't MMW handle whatever paths the host OS can support--so e.g. if Win8 supports paths > 260 chars, shouldn't MMW do the same? And if e.g. XP supports a max of 260 chars, then on XP MM shouldn't scan > 260 chars ? |
|
|
Win8 doesn't support paths longer than 260 chars, but the whole story is much more complicated, see http://msdn.microsoft.com/en-us/library/aa365247%28VS.85%29.aspx and similar. Namely: a) We could use "\\?\" prefix, but it varies on file system used (NTFS, FAT, FlexRAIDFS, ...), many but not all file I/O APIs support "\\?\"; you need look at the reference topic for each API to be sure b) 8.3 aliasing also isn't a good way, not all file systems follow the tilde substitution convention, and systems can be configured to disable 8.3 alias generation even if they normally support it Based on the info I've grabbled there really isn't a general solution how to handle long paths on Windows, therefore I would suggest to not scan paths longer than 260 into library at all to avoid all the possible issues. |
|
|
Well, I would not go to that approach and to confirm this I used http://www.norm-online.net/FindFullFileNameGreaterThan255.php Ended with 784 Files (676 were Music) and 32k+ of Files on My NAS. Re: a) I agree, We should try and if it fail than we know how to proceed. This will be more and more relevant for MMW 5.0+ b) Even when it is disabled Delphi create 8.3 paths and we already use it for plugins and in Scripting API I agree that there is no standard solution and will never be till Microsoft stop backward compatibility and draw the line Linux, Mac do not have that issue. Just for test created 5k path On NAS without problem and it also included Unicode characters. EDIT NOTE: ALL UNC Paths start with "\\" Which make full filenames should work according to spec. |
|
|
It seems that for some users there is a crash while scanning FLAC files with path longer than 260 chars, see ticket FZM-362109 for details. I wonder whether it could bee related to the new FLAC plugin created by Michal, to find more... |
|
|
Note regarding Ticket FZM-362109: Last scanned file have filename of 271 Chars "E:\Musik\Musik\Gustav Mahler\Mahler - Sinf. Nr. 2\Mahler Sinf. Nr. 2 Auferstehung CD 1\6 - Symphony No. 2 'Resurrection'. Movement 1- Ausdrucksvoll (English horn & bass clarinet)Symphony No. 2 'Resurrection'. Movement 1- Ausdrucksvoll (English horn & bass clarinet).flac" |
|
|
Assigned to Michal to check FLAC Plugin. |
|
|
Peke, before Michal review this, can you reproduce the issue? We haven't been ble to replicate it before (0011799:0039408) and we suspected that it is related to file system used (FlexRAIDFS) on user's external drive where he copied music from MAC. Is this the same also for the other users experiencing this issue? It would be very useful to find a way how to reproduce the issue, i.e. is it really file system used on the particular drive? Isn't it e.g. that user has disable 8.3 aliases on his system or a similar settings related to file paths that would help us to reproduce it? Could you collect more info/links to users experiencing this issue? It sounds like a regression in 4.1 for those who can replicate this issue? |
|
|
FLAC plugin uses TStreamAccess from the main application and does not handle file path at all. I've tried the same long filepath and file name (you can create it e.g. by using "subst" command and rename the file in the shorter substituted path) and everything is ok, I can read tag, write tag and play the file. |
|
|
This is nothing to do with "disable 8.3 aliases" as delphi command calculate it internally it is file system related and possible bug in windows where you can force creation of such long path, but than you can't access it, it happened to me several times. Like Michal pointed there is always an workaround like in case of subst. Also I had big issue last night where my NAS created filename "http://www.microsoft.com.html" that crash all SAMBA access untill I deleted it from Web interface. |
|
|
@Support team, Due to increasing number of reports, I wonder, is this a regression in 4.1.0 or not ??? i.e. Did MM 4.0.7 also crash for the users while scanning >260 chars long paths or it started after upgrade to 4.1? Because we cannot reproduce it and it is not clear why MM crashes we should get as much info as possible, for example if we found that it is a regression then we should review related issues that could had an impact on it: 0010280 (build 1623) 0010626 (build 1628) 0010692 (build 1630) i.e. for the users that can reproduce it would be valuable to find whether it started in build 1623, 1628 or 1630. If it is not a regression (or maybe also in case it is) then I think that we could skip scanning of paths longer than 260 chars in MM 4.1.1 to prevent MM from crashing and list them as files not matching import criteria in the scan log (+ add KB article), as such a files can always have issues on Windows, the same limitation we have implemented when renaming/copying/moving files, i.e. MM does not allow to create such a long paths at all. |
|
|
I was unable to Replicate crash. Neither test MMW versions Add tracks to library (1623, 1628, 1630, 1693). To test Playback I added one by one using "File -> Open URL or File...." and all played all 4 files. as of 1630 MMW started to complain that network path is unavailable on playback start (Same Behavior in 1693), I saved MediaMonkey.M3U8 for easier test Drag and Drop from Windows Explorer do not work in 1630 (It works in 1693) and MMW get incorrect filename. I used 1630 as baseline to create Debug Log I also Created Log from 1693. Logs are uploaded to FTP. NOTE: MMW 4.0.7 logs are also uploaded |
|
|
Peke, I guess that your debug logs are useless once you cannot replicate it. I wanted you to get touch with the users that can replicate the crash to find whether it is a regression or not. |
|
|
User do not observe crash anymore. Problem was that Error dialog from log files show in background and user thought that MMW crashed. I wonder why that Dialog even show in the first place as track plays correctly which basically means that MMW correctly handle the files but not in all places? |
|
|
OK, so what is that error dialog, do we have a screenshot? BTW: I haven't found any ELF related to this kind of crash so it would imply that there isn't crash, but a hidden dialog. Is the regression that MM shows the error dialog unlike 4.0.7? Assigned back to you to get this info from users (especially ELF could help us a lot). |
|
|
yes Screenshot is inside archive along with 1630 LOGs. There is no ELF created as Dialog is not actually error but announcement that network path is unavailable for Tag reading followed immediately by dialog Operation finished successful, after clicking on Close File Starts to play. User observes crash renamed all other files and MMW started to work. All users supplied LOG files in tickets where last scanned file was too long and as soon as we asked for rename all worked normally. I'm unable to replicate crash. I asked users for MediaMonkey.elf if existing. |
|
|
Re the status of this issue, based on offline and IM discussions with Peke this issue doesn't seem to be a regression. Regressions were items 2,3,4 that were fixed before MM 4.1.0 release. Items 5,6 were always issues (even if 4.0.7) and are Windows issues (not fixable). Peke indicated that the issue 1) was rather freeze and (not crash) when he made the remote session. As for the skipping files longer than 260 chars during scans, this would probably result in disatisfaction, because as Peke noted he has 32K tracks on his NAS with paths longer than 260 chars, so I would rather avoid to do this change for 4.1.1 |
|
|
To clarify that there is no regressions and to do more direct example of bug and MMW handling I created small video of parallel testing of MMW 4.0.7 and 4.1.1 which clearly show that there was no regressions in 4.1 along that 4.1 fixes improved MMW handling of such files, but also show that behavior even consistent over the versions is not complete solution for handling long filenames which should be fixed. Files used to test nd create video: "c:\Users\Peke\Desktop\MediaMonkey bug testing\Music\organized music into folders to test long filenames\Audiobooks\Testing folder for bugs and regressions\Arthur C. Clarke & Gentry Lee - All 6 Rama Series Audio Books\Lady in Red Vol. 1\Linda Ronstadt & Aaron Neville\12. Linda Ronstadt & Aaron Neville - Don't Know Much\12. Communards - Don't Leave Me This Way (Radio Edit).flac" File used in test regarding Tag reading of MP3s and error dialog: "c:\Users\Peke\Desktop\MediaMonkey bug testing\Music\organized music into folders to test long filenames\Audiobooks\Testing folder for bugs and regressions\Arthur C. Clarke & Gentry Lee - All 6 Rama Series Audio Books\Lady in Red Vol. 1\Linda Ronstadt & Aaron Neville\12. Linda Ronstadt & Aaron Neville - Don't Know Much\04-carly_rae_jepsen-tell_me.mp3" File is uploaded to FTP. |
|
|
Another user (from ticket JLV-445292) observes MM crash while scanning 268 characters long filename. I've just re-tested and found that MM 4.0.7 doesn't scan such a long filenames at all (reports "Error while reading file info" in the log), 4.1.0 scans even long filenames. So it is a regression, MM should behave like in 4.0.7 and should skip these files during scans, just finding which change caused it to fix it properly. |
|
|
Fixed in build 4.1.1.1695 Because we can hardly know why it crashes on some systems (no crash log and it crashes in try ... except block) and because Windows always have issues with long paths (see my note 0011799:0039472 for details) the fix looks like this: if the path is longer than MAX_PATH on Windows (260 chars) and it is path from local or removable drive (e.g. D:\path.mp3) then such a path is skipped as "doesn't match import criteria", note that in 4.0.7 it was skipped as "error reading file info" because 4.0.7 couldn't handle long paths at all (0008020). Note that for network paths (//...) it still accepts paths longer than MAX_PATH so it shouldn't affect tracks on NAS etc. Note that the tracks can be still D&D onto MM or opened via Open File, they are just skipped during scans. |
|
|
Verified 1696 File scans are same as in 4.0.7 |
