Changeset 240969 in webkit


Ignore:
Timestamp:
Feb 5, 2019 2:49:12 AM (5 years ago)
Author:
zandobersek@gmail.com
Message:

[GLib] Stop URI-escaping file system representations
https://bugs.webkit.org/show_bug.cgi?id=194213

Reviewed by Carlos Garcia Campos.

Stop URI-escaping of file representation strings in
FileSystem::stringFromFileSystemRepresentation(), and URI-unescaping
of strings in FileSystem::fileSystemRepresentation().

This behavior deviates from POSIX and CF implementations and is
currently breaking IndexedDB-specific calculation of database sizes due
to directory components used in that process that are URL-based and are
as such URI-escaped. When unescaped, those single directory components
explode into multiple directory components, leading to incorrect total
database size calculation when iterating the database directory.

FileSystem::stringFromFileSystemRepresentation() now retrieves GLib's
filename charsets and in worst case converts the filesystem
representation to UTF-8 before String::fromUTF8() is used.
FileSystem::fileSystemRepresentation() reverses that process, taking
String's UTF-8 data and converting it to target charset if necessary.

Other FileSystem functions are adjusted to convert passed-in String
objects to filesystem representations.

  • wtf/glib/FileSystemGlib.cpp:

(WTF::FileSystemImpl::stringFromFileSystemRepresentation):
(WTF::FileSystemImpl::fileSystemRepresentation):
(WTF::FileSystemImpl::validRepresentation):
(WTF::FileSystemImpl::filenameForDisplay):
(WTF::FileSystemImpl::fileExists):
(WTF::FileSystemImpl::deleteFile):
(WTF::FileSystemImpl::deleteEmptyDirectory):
(WTF::FileSystemImpl::getFileStat):
(WTF::FileSystemImpl::getFileLStat):
(WTF::FileSystemImpl::makeAllDirectories):
(WTF::FileSystemImpl::createSymbolicLink):
(WTF::FileSystemImpl::pathGetFileName):
(WTF::FileSystemImpl::getVolumeFreeSpace):
(WTF::FileSystemImpl::directoryName):
(WTF::FileSystemImpl::listDirectory):
(WTF::FileSystemImpl::openFile):
(WTF::FileSystemImpl::moveFile):
(WTF::FileSystemImpl::hardLinkOrCopyFile):
(WTF::FileSystemImpl::getFileDeviceId): Align with POSIX implementation
and treat input CString as an existing filesystem representation.
(WTF::FileSystemImpl::unescapedFilename): Deleted.

Location:
trunk/Source/WTF
Files:
2 edited

Legend:

Unmodified
Added
Removed
  • trunk/Source/WTF/ChangeLog

    r240962 r240969  
     12019-02-05  Zan Dobersek  <zdobersek@igalia.com>
     2
     3        [GLib] Stop URI-escaping file system representations
     4        https://bugs.webkit.org/show_bug.cgi?id=194213
     5
     6        Reviewed by Carlos Garcia Campos.
     7
     8        Stop URI-escaping of file representation strings in
     9        FileSystem::stringFromFileSystemRepresentation(), and URI-unescaping
     10        of strings in FileSystem::fileSystemRepresentation().
     11
     12        This behavior deviates from POSIX and CF implementations and is
     13        currently breaking IndexedDB-specific calculation of database sizes due
     14        to directory components used in that process that are URL-based and are
     15        as such URI-escaped. When unescaped, those single directory components
     16        explode into multiple directory components, leading to incorrect total
     17        database size calculation when iterating the database directory.
     18
     19        FileSystem::stringFromFileSystemRepresentation() now retrieves GLib's
     20        filename charsets and in worst case converts the filesystem
     21        representation to UTF-8 before String::fromUTF8() is used.
     22        FileSystem::fileSystemRepresentation() reverses that process, taking
     23        String's UTF-8 data and converting it to target charset if necessary.
     24
     25        Other FileSystem functions are adjusted to convert passed-in String
     26        objects to filesystem representations.
     27
     28        * wtf/glib/FileSystemGlib.cpp:
     29        (WTF::FileSystemImpl::stringFromFileSystemRepresentation):
     30        (WTF::FileSystemImpl::fileSystemRepresentation):
     31        (WTF::FileSystemImpl::validRepresentation):
     32        (WTF::FileSystemImpl::filenameForDisplay):
     33        (WTF::FileSystemImpl::fileExists):
     34        (WTF::FileSystemImpl::deleteFile):
     35        (WTF::FileSystemImpl::deleteEmptyDirectory):
     36        (WTF::FileSystemImpl::getFileStat):
     37        (WTF::FileSystemImpl::getFileLStat):
     38        (WTF::FileSystemImpl::makeAllDirectories):
     39        (WTF::FileSystemImpl::createSymbolicLink):
     40        (WTF::FileSystemImpl::pathGetFileName):
     41        (WTF::FileSystemImpl::getVolumeFreeSpace):
     42        (WTF::FileSystemImpl::directoryName):
     43        (WTF::FileSystemImpl::listDirectory):
     44        (WTF::FileSystemImpl::openFile):
     45        (WTF::FileSystemImpl::moveFile):
     46        (WTF::FileSystemImpl::hardLinkOrCopyFile):
     47        (WTF::FileSystemImpl::getFileDeviceId): Align with POSIX implementation
     48        and treat input CString as an existing filesystem representation.
     49        (WTF::FileSystemImpl::unescapedFilename): Deleted.
     50
    1512019-02-04  Ms2ger  <Ms2ger@igalia.com>
    252
  • trunk/Source/WTF/wtf/glib/FileSystemGlib.cpp

    r240437 r240969  
    3535#include <wtf/glib/GRefPtr.h>
    3636#include <wtf/glib/GUniquePtr.h>
     37#include <wtf/text/ASCIIFastPath.h>
    3738#include <wtf/text/CString.h>
    3839#include <wtf/text/StringBuilder.h>
     
    4344namespace FileSystemImpl {
    4445
    45 /* On linux file names are just raw bytes, so also strings that cannot be encoded in any way
    46  * are valid file names. This mean that we cannot just store a file name as-is in a String
    47  * but we have to escape it.
    48  * On Windows the GLib file name encoding is always UTF-8 so we can optimize this case. */
    49 String stringFromFileSystemRepresentation(const char* fileSystemRepresentation)
    50 {
    51     if (!fileSystemRepresentation)
    52         return String();
    53 
    54 #if OS(WINDOWS)
    55     return String::fromUTF8(fileSystemRepresentation);
    56 #else
    57     GUniquePtr<gchar> escapedString(g_uri_escape_string(fileSystemRepresentation, "/:", FALSE));
    58     return escapedString.get();
    59 #endif
    60 }
    61 
    62 static GUniquePtr<char> unescapedFilename(const String& path)
     46String stringFromFileSystemRepresentation(const char* representation)
     47{
     48    if (!representation)
     49        return { };
     50
     51    // Short-cut to String creation when only ASCII characters are used.
     52    size_t representationLength = strlen(representation);
     53    if (charactersAreAllASCII(reinterpret_cast<const LChar*>(representation), representationLength))
     54        return String(representation, representationLength);
     55
     56    // If the returned charset is UTF-8 (i.e. g_get_filename_charsets() returns true),
     57    // go directly to String creation.
     58    const gchar** filenameCharsets = nullptr;
     59    if (g_get_filename_charsets(&filenameCharsets))
     60        return String::fromUTF8(representation, representationLength);
     61
     62    ASSERT(filenameCharsets);
     63    // FIXME: If possible, we'd want to convert directly to UTF-16 and construct
     64    // WTF::String object with resulting data.
     65    size_t utf8Length = 0;
     66    GUniquePtr<gchar> utf8(g_convert(representation, representationLength,
     67        "UTF-8", filenameCharsets[0], nullptr, &utf8Length, nullptr));
     68    if (!utf8)
     69        return { };
     70
     71    return String::fromUTF8(utf8.get(), utf8Length);
     72}
     73
     74CString fileSystemRepresentation(const String& path)
    6375{
    6476    if (path.isEmpty())
    65         return nullptr;
    66 #if OS(WINDOWS)
    67     return GUniquePtr<char>(g_strdup(path.utf8().data()));
    68 #else
    69     return GUniquePtr<char>(g_uri_unescape_string(path.utf8().data(), nullptr));
    70 #endif
    71 }
    72 
    73 CString fileSystemRepresentation(const String& path)
    74 {
    75 #if OS(WINDOWS)
    76     return path.utf8();
    77 #else
    78     GUniquePtr<gchar> filename = unescapedFilename(path);
    79     return filename.get();
    80 #endif
     77        return { };
     78
     79    CString utf8 = path.utf8();
     80
     81    // If the returned charset is UTF-8 (i.e. g_get_filename_charsets() returns true),
     82    // simply return the CString object.
     83    const gchar** filenameCharsets = nullptr;
     84    if (g_get_filename_charsets(&filenameCharsets))
     85        return utf8;
     86
     87    ASSERT(filenameCharsets);
     88    // FIXME: If possible, we'd want to convert directly from WTF::String's UTF-16 data.
     89    size_t representationLength = 0;
     90    GUniquePtr<gchar> representation(g_convert(utf8.data(), utf8.length(),
     91        filenameCharsets[0], "UTF-8", nullptr, &representationLength, nullptr));
     92    if (!representation)
     93        return { };
     94
     95    return CString(representation.get(), representationLength);
     96}
     97
     98bool validRepresentation(const CString& representation)
     99{
     100    auto* data = representation.data();
     101    return !!data && data[0] != '\0';
    81102}
    82103
     
    87108    return string;
    88109#else
    89     GUniquePtr<gchar> filename = unescapedFilename(string);
    90     if (!filename)
     110    auto filename = fileSystemRepresentation(string);
     111    if (!validRepresentation(filename))
    91112        return string;
    92113
    93     GUniquePtr<gchar> display(g_filename_to_utf8(filename.get(), -1, nullptr, nullptr, nullptr));
     114    GUniquePtr<gchar> display(g_filename_display_name(filename.data()));
    94115    if (!display)
    95116        return string;
    96 
    97117    return String::fromUTF8(display.get());
    98118#endif
     
    101121bool fileExists(const String& path)
    102122{
    103     GUniquePtr<gchar> filename = unescapedFilename(path);
    104     return filename ? g_file_test(filename.get(), G_FILE_TEST_EXISTS) : false;
     123    auto filename = fileSystemRepresentation(path);
     124    return validRepresentation(filename) ? g_file_test(filename.data(), G_FILE_TEST_EXISTS) : false;
    105125}
    106126
    107127bool deleteFile(const String& path)
    108128{
    109     GUniquePtr<gchar> filename = unescapedFilename(path);
    110     return filename ? g_remove(filename.get()) != -1 : false;
     129    auto filename = fileSystemRepresentation(path);
     130    return validRepresentation(filename) ? g_remove(filename.data()) != -1 : false;
    111131}
    112132
    113133bool deleteEmptyDirectory(const String& path)
    114134{
    115     GUniquePtr<gchar> filename = unescapedFilename(path);
    116     return filename ? g_rmdir(filename.get()) != -1 : false;
     135    auto filename = fileSystemRepresentation(path);
     136    return validRepresentation(filename) ? g_rmdir(filename.data()) != -1 : false;
    117137}
    118138
    119139static bool getFileStat(const String& path, GStatBuf* statBuffer)
    120140{
    121     GUniquePtr<gchar> filename = unescapedFilename(path);
    122     if (!filename)
    123         return false;
    124 
    125     return g_stat(filename.get(), statBuffer) != -1;
     141    auto filename = fileSystemRepresentation(path);
     142    if (!validRepresentation(filename))
     143        return false;
     144
     145    return g_stat(filename.data(), statBuffer) != -1;
    126146}
    127147
    128148static bool getFileLStat(const String& path, GStatBuf* statBuffer)
    129149{
    130     GUniquePtr<gchar> filename = unescapedFilename(path);
    131     if (!filename)
    132         return false;
    133 
    134     return g_lstat(filename.get(), statBuffer) != -1;
     150    auto filename = fileSystemRepresentation(path);
     151    if (!validRepresentation(filename))
     152        return false;
     153
     154    return g_lstat(filename.data(), statBuffer) != -1;
    135155}
    136156
     
    226246bool makeAllDirectories(const String& path)
    227247{
    228     GUniquePtr<gchar> filename = unescapedFilename(path);
    229     return filename ? g_mkdir_with_parents(filename.get(), S_IRWXU) != -1 : false;
     248    auto filename = fileSystemRepresentation(path);
     249    return validRepresentation(filename) ? g_mkdir_with_parents(filename.data(), S_IRWXU) != -1 : false;
    230250}
    231251
     
    238258{
    239259    CString targetPathFSRep = fileSystemRepresentation(targetPath);
    240     if (!targetPathFSRep.data() || targetPathFSRep.data()[0] == '\0')
     260    if (!validRepresentation(targetPathFSRep))
    241261        return false;
    242262
    243263    CString symbolicLinkPathFSRep = fileSystemRepresentation(symbolicLinkPath);
    244     if (!symbolicLinkPathFSRep.data() || symbolicLinkPathFSRep.data()[0] == '\0')
     264    if (!validRepresentation(symbolicLinkPathFSRep))
    245265        return false;
    246266
     
    248268}
    249269
    250 String pathGetFileName(const String& pathName)
    251 {
    252     GUniquePtr<gchar> tmpFilename = unescapedFilename(pathName);
    253     if (!tmpFilename)
    254         return pathName;
    255 
    256     GUniquePtr<gchar> baseName(g_path_get_basename(tmpFilename.get()));
     270String pathGetFileName(const String& path)
     271{
     272    auto filename = fileSystemRepresentation(path);
     273    if (!validRepresentation(filename))
     274        return path;
     275
     276    GUniquePtr<gchar> baseName(g_path_get_basename(filename.data()));
    257277    return String::fromUTF8(baseName.get());
    258278}
     
    260280bool getVolumeFreeSpace(const String& path, uint64_t& freeSpace)
    261281{
    262     GUniquePtr<gchar> filename = unescapedFilename(path);
    263     if (!filename)
    264         return false;
    265 
    266     GRefPtr<GFile> file = adoptGRef(g_file_new_for_path(filename.get()));
     282    auto filename = fileSystemRepresentation(path);
     283    if (!validRepresentation(filename))
     284        return false;
     285
     286    GRefPtr<GFile> file = adoptGRef(g_file_new_for_path(filename.data()));
    267287    GRefPtr<GFileInfo> fileInfo = adoptGRef(g_file_query_filesystem_info(file.get(), G_FILE_ATTRIBUTE_FILESYSTEM_FREE, 0, 0));
    268288    if (!fileInfo)
     
    275295String directoryName(const String& path)
    276296{
    277     GUniquePtr<gchar> filename = unescapedFilename(path);
    278     if (!filename)
     297    auto filename = fileSystemRepresentation(path);
     298    if (!validRepresentation(filename))
    279299        return String();
    280300
    281     GUniquePtr<char> dirname(g_path_get_dirname(filename.get()));
     301    GUniquePtr<char> dirname(g_path_get_dirname(filename.data()));
    282302    return String::fromUTF8(dirname.get());
    283303}
     
    287307    Vector<String> entries;
    288308
    289     GUniquePtr<gchar> filename = unescapedFilename(path);
    290     if (!filename)
     309    auto filename = fileSystemRepresentation(path);
     310    if (!validRepresentation(filename))
    291311        return entries;
    292312
    293     GUniquePtr<GDir> dir(g_dir_open(filename.get(), 0, nullptr));
     313    GUniquePtr<GDir> dir(g_dir_open(filename.data(), 0, nullptr));
    294314    if (!dir)
    295315        return entries;
     
    300320            continue;
    301321
    302         GUniquePtr<gchar> entry(g_build_filename(filename.get(), name, nullptr));
     322        GUniquePtr<gchar> entry(g_build_filename(filename.data(), name, nullptr));
    303323        entries.append(stringFromFileSystemRepresentation(entry.get()));
    304324    }
     
    321341PlatformFileHandle openFile(const String& path, FileOpenMode mode)
    322342{
    323     GUniquePtr<gchar> filename = unescapedFilename(path);
    324     if (!filename)
     343    auto filename = fileSystemRepresentation(path);
     344    if (!validRepresentation(filename))
    325345        return invalidPlatformFileHandle;
    326346
    327     GRefPtr<GFile> file = adoptGRef(g_file_new_for_path(filename.get()));
     347    GRefPtr<GFile> file = adoptGRef(g_file_new_for_path(filename.data()));
    328348    GFileIOStream* ioStream = 0;
    329349    if (mode == FileOpenMode::Read)
    330350        ioStream = g_file_open_readwrite(file.get(), 0, 0);
    331351    else if (mode == FileOpenMode::Write) {
    332         if (g_file_test(filename.get(), static_cast<GFileTest>(G_FILE_TEST_EXISTS | G_FILE_TEST_IS_REGULAR)))
     352        if (g_file_test(filename.data(), static_cast<GFileTest>(G_FILE_TEST_EXISTS | G_FILE_TEST_IS_REGULAR)))
    333353            ioStream = g_file_open_readwrite(file.get(), 0, 0);
    334354        else
     
    396416bool moveFile(const String& oldPath, const String& newPath)
    397417{
    398     GUniquePtr<gchar> oldFilename = unescapedFilename(oldPath);
    399     if (!oldFilename)
    400         return false;
    401 
    402     GUniquePtr<gchar> newFilename = unescapedFilename(newPath);
    403     if (!newFilename)
    404         return false;
    405 
    406     GRefPtr<GFile> oldFile = adoptGRef(g_file_new_for_path(oldFilename.get()));
    407     GRefPtr<GFile> newFile = adoptGRef(g_file_new_for_path(newFilename.get()));
     418    auto oldFilename = fileSystemRepresentation(oldPath);
     419    if (!validRepresentation(oldFilename))
     420        return false;
     421
     422    auto newFilename = fileSystemRepresentation(newPath);
     423    if (!validRepresentation(newFilename))
     424        return false;
     425
     426    GRefPtr<GFile> oldFile = adoptGRef(g_file_new_for_path(oldFilename.data()));
     427    GRefPtr<GFile> newFile = adoptGRef(g_file_new_for_path(newFilename.data()));
    408428
    409429    return g_file_move(oldFile.get(), newFile.get(), G_FILE_COPY_OVERWRITE, nullptr, nullptr, nullptr, nullptr);
     
    415435    return !!::CopyFile(source.charactersWithNullTermination().data(), destination.charactersWithNullTermination().data(), TRUE);
    416436#else
    417     GUniquePtr<gchar> sourceFilename = unescapedFilename(source);
    418     if (!sourceFilename)
    419         return false;
    420 
    421     GUniquePtr<gchar> destinationFilename = unescapedFilename(destination);
    422     if (!destinationFilename)
    423         return false;
    424 
    425     if (!link(sourceFilename.get(), destinationFilename.get()))
     437    auto sourceFilename = fileSystemRepresentation(source);
     438    if (!validRepresentation(sourceFilename))
     439        return false;
     440
     441    auto destinationFilename = fileSystemRepresentation(destination);
     442    if (!validRepresentation(destinationFilename))
     443        return false;
     444
     445    if (!link(sourceFilename.data(), destinationFilename.data()))
    426446        return true;
    427447
    428448    // Hard link failed. Perform a copy instead.
    429     GRefPtr<GFile> sourceFile = adoptGRef(g_file_new_for_path(sourceFilename.get()));
    430     GRefPtr<GFile> destinationFile = adoptGRef(g_file_new_for_path(destinationFilename.get()));
     449    GRefPtr<GFile> sourceFile = adoptGRef(g_file_new_for_path(sourceFilename.data()));
     450    GRefPtr<GFile> destinationFile = adoptGRef(g_file_new_for_path(destinationFilename.data()));
    431451    return g_file_copy(sourceFile.get(), destinationFile.get(), G_FILE_COPY_NONE, nullptr, nullptr, nullptr, nullptr);
    432452#endif
     
    435455Optional<int32_t> getFileDeviceId(const CString& fsFile)
    436456{
    437     GUniquePtr<gchar> filename = unescapedFilename(fsFile.data());
    438     if (!filename)
    439         return WTF::nullopt;
    440 
    441     GRefPtr<GFile> file = adoptGRef(g_file_new_for_path(filename.get()));
     457    GRefPtr<GFile> file = adoptGRef(g_file_new_for_path(fsFile.data()));
    442458    GRefPtr<GFileInfo> fileInfo = adoptGRef(g_file_query_filesystem_info(file.get(), G_FILE_ATTRIBUTE_UNIX_DEVICE, nullptr, nullptr));
    443459    if (!fileInfo)
Note: See TracChangeset for help on using the changeset viewer.