The goal is to refactor the file I/O code to make it easier to re-use.
TiddlyWiki ships with FileSystem? modules for Internet Explorer, Firefox, Safari and Opera/Java that implement file saving and loading, and a set of UTF-8 conversion functions. There's a thin abstraction layer that does some really simple (and not particularly extensible) sniffing to figure out the prevailing FileSystem? type.
The main driver for the work is the way that the current implementation doesn't attempt to hide a fundamental issue with the Internet Explorer implementation, based on the FileSystem? !ActiveX object. It utterly fails to understand UTF-8 when reading a file, and instead sees each byte as an ANSI character. The JavaScript glue around the !ActiveX object dutifully converts the ANSI characters to their equivalent Unicode characters which for all but the core 96 or so ASCII characters will be pretty meaningless. (Note in particular that the Unicode equivalent may not have the same code as the ANSI original). The trick that TiddlyWiki uses is that the FileSystem? object does precisely the opposite thing when saving - from a UTF-8 perspective, it converts the Unicode characters back to their equivalent ANSI codes and writes them out, in the process assembling the original UTF-8 byte sequence.
A further complication is that attempting to save an arbitrary Unicode JavaScript string is tricky. Internet Explorer's FileSystem? object will complain if it sees any Unicode characters that don't map to an ANSII equivalent, so you have to use an HTML or JavaScript Unicode escaping mechanism to express non-ANSI Unicode characters (eg \u2345).
Other goals are:
- to do the FileSystem? sniffing better, particularly to give better error messages when users attempt to save a TiddlyWiki that was loaded over http:
- to incorporate the logic for parsing file:/// URIs, including consistent handling for relative pathnames
If nothing else, the refactoring should result in better documentation about this peculiarity...