Changeset 152976 in webkit


Ignore:
Timestamp:
Jul 22, 2013 9:35:15 AM (11 years ago)
Author:
commit-queue@webkit.org
Message:

[Curl] Download fails for certain urls.
https://bugs.webkit.org/show_bug.cgi?id=118468

Patch by peavo@outlook.com <peavo@outlook.com> on 2013-07-22
Reviewed by Brent Fulgham.

Source/WebCore:

The current Curl download implementation has a few shortcomings:
1) Downloading from secure locations fails. We need to provide Curl with the path to a certificate file (.pem file).
2) Cookies are not set in the download request. We need to give Curl the path to the cookie file.
3) When a normal load is converted to a download, some of the headers from the original request is not sent (e.g. Referer, User agent).

  • platform/network/curl/CurlDownload.cpp:

(WebCore::CurlDownload::CurlDownload): Initialize custom headers member.
(WebCore::CurlDownload::~CurlDownload): Free custom headers member.
(WebCore::CurlDownload::init): Set certificate and cookie file path.
(WebCore::CurlDownload::closeFile): Check file handle against value for invalid platform handle.
(WebCore::CurlDownload::writeDataToFile): Added utility method to write download data to file.
(WebCore::CurlDownload::addHeaders): Added utility method to add headers to request.
(WebCore::CurlDownload::didReceiveData): Use writeDataToFile utility method.

  • platform/network/curl/CurlDownload.h:

Put class in WebCore namespace.
Added method to init download from resource handle, request, and response object.
Added utility method to write download data to file.
Added utility method to add headers to request.
Added custom headers member.

Source/WebKit/win:

Initialize download from provided resource handle, request, and response object.

  • WebDownload.h: Use WebCore namespace for Curl download class.
  • WebDownloadCurl.cpp:

(WebDownload::init): Initialize download from provided resource handle, request, and response object.

Location:
trunk/Source
Files:
6 edited

Legend:

Unmodified
Added
Removed
  • trunk/Source/WebCore/ChangeLog

    r152975 r152976  
     12013-07-22  peavo@outlook.com  <peavo@outlook.com>
     2
     3        [Curl] Download fails for certain urls.
     4        https://bugs.webkit.org/show_bug.cgi?id=118468
     5
     6        Reviewed by Brent Fulgham.
     7
     8        The current Curl download implementation has a few shortcomings:
     9        1) Downloading from secure locations fails. We need to provide Curl with the path to a certificate file (.pem file).
     10        2) Cookies are not set in the download request. We need to give Curl the path to the cookie file.
     11        3) When a normal load is converted to a download, some of the headers from the original request is not sent (e.g. Referer, User agent).
     12
     13        * platform/network/curl/CurlDownload.cpp:
     14        (WebCore::CurlDownload::CurlDownload): Initialize custom headers member.
     15        (WebCore::CurlDownload::~CurlDownload): Free custom headers member.
     16        (WebCore::CurlDownload::init): Set certificate and cookie file path.
     17        (WebCore::CurlDownload::closeFile): Check file handle against value for invalid platform handle.
     18        (WebCore::CurlDownload::writeDataToFile): Added utility method to write download data to file.
     19        (WebCore::CurlDownload::addHeaders): Added utility method to add headers to request.
     20        (WebCore::CurlDownload::didReceiveData): Use writeDataToFile utility method.
     21        * platform/network/curl/CurlDownload.h:
     22        Put class in WebCore namespace.
     23        Added method to init download from resource handle, request, and response object.
     24        Added utility method to write download data to file.
     25        Added utility method to add headers to request.
     26        Added custom headers member.
     27
    1282013-07-22  Eric Carlson  <eric.carlson@apple.com>
    229
  • trunk/Source/WebCore/platform/network/curl/CurlDownload.cpp

    r152649 r152976  
    2929#include <WebCore/HTTPParsers.h>
    3030#include <WebCore/MainThreadTask.h>
     31#include <WebCore/ResourceRequest.h>
    3132
    3233#include <wtf/MainThread.h>
     
    3738template<> struct CrossThreadCopierBase<false, false, CurlDownload*> : public CrossThreadCopierPassThrough<CurlDownload*> {
    3839};
     40
     41namespace WebCore {
    3942
    4043// CurlDownloadManager -------------------------------------------------------------------
     
    214217CurlDownload::CurlDownload()
    215218: m_curlHandle(0)
     219, m_customHeaders(0)
    216220, m_url(0)
    217221, m_tempHandle(invalidPlatformFileHandle)
     
    227231    if (m_url)
    228232        fastFree(m_url);
     233
     234    if (m_customHeaders)
     235        curl_slist_free_all(m_customHeaders);
    229236
    230237    closeFile();
     
    254261    curl_easy_setopt(m_curlHandle, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
    255262
     263    const char* certPath = getenv("CURL_CA_BUNDLE_PATH");
     264    if (certPath)
     265        curl_easy_setopt(m_curlHandle, CURLOPT_CAINFO, certPath);
     266
     267    const char* cookieJarPath = getenv("CURL_COOKIE_JAR_PATH");
     268    if (cookieJarPath)
     269        curl_easy_setopt(m_curlHandle, CURLOPT_COOKIEFILE, cookieJarPath);
     270
    256271    m_listener = listener;
    257272}
    258273
     274void CurlDownload::init(CurlDownloadListener* listener, ResourceHandle*, const ResourceRequest& request, const ResourceResponse&)
     275{
     276    if (!listener)
     277        return;
     278
     279    MutexLocker locker(m_mutex);
     280
     281    KURL url(ParsedURLString, request.url());
     282
     283    init(listener, url);
     284
     285    addHeaders(request);
     286}
     287
    259288bool CurlDownload::start()
    260289{
     
    289318    MutexLocker locker(m_mutex);
    290319
    291     if (m_tempHandle) {
     320    if (m_tempHandle != invalidPlatformFileHandle) {
    292321        WebCore::closeFile(m_tempHandle);
    293322        m_tempHandle = invalidPlatformFileHandle;
     
    301330
    302331    ::MoveFileEx(m_tempPath.charactersWithNullTermination().data(), m_destination.charactersWithNullTermination().data(), MOVEFILE_COPY_ALLOWED | MOVEFILE_REPLACE_EXISTING);
     332}
     333
     334void CurlDownload::writeDataToFile(const char* data, int size)
     335{
     336    if (m_tempPath.isEmpty())
     337        m_tempPath = openTemporaryFile("download", m_tempHandle);
     338
     339    if (m_tempHandle != invalidPlatformFileHandle)
     340        writeToFile(m_tempHandle, data, size);
     341}
     342
     343void CurlDownload::addHeaders(const ResourceRequest& request)
     344{
     345    if (request.httpHeaderFields().size() > 0) {
     346        struct curl_slist* headers = 0;
     347
     348        HTTPHeaderMap customHeaders = request.httpHeaderFields();
     349        HTTPHeaderMap::const_iterator end = customHeaders.end();
     350        for (HTTPHeaderMap::const_iterator it = customHeaders.begin(); it != end; ++it) {
     351            const String& value = it->value;
     352            String headerString(it->key);
     353            if (value.isEmpty())
     354                // Insert the ; to tell curl that this header has an empty value.
     355                headerString.append(";");
     356            else {
     357                headerString.append(": ");
     358                headerString.append(value);
     359            }
     360            CString headerLatin1 = headerString.latin1();
     361            headers = curl_slist_append(headers, headerLatin1.data());
     362        }
     363
     364        if (headers) {
     365            curl_easy_setopt(m_curlHandle, CURLOPT_HTTPHEADER, headers);
     366            m_customHeaders = headers;
     367        }
     368    }
    303369}
    304370
     
    336402    callOnMainThread<CurlDownload*, CurlDownload*, int, int>(receivedDataCallback, this, size);
    337403
    338     if (m_tempPath.isEmpty())
    339         m_tempPath = openTemporaryFile("download", m_tempHandle);
    340 
    341     if (m_tempHandle != invalidPlatformFileHandle) {
    342         const char* fileData = static_cast<const char*>(data);
    343         writeToFile(m_tempHandle, fileData, size);
    344     }
     404    writeDataToFile(static_cast<const char*>(data), size);
    345405}
    346406
     
    426486        download->didReceiveResponse();
    427487}
     488
     489}
  • trunk/Source/WebCore/platform/network/curl/CurlDownload.h

    r152297 r152976  
    2828
    2929#include <WebCore/FileSystem.h>
     30#include <WebCore/ResourceHandle.h>
    3031#include <WebCore/ResourceResponse.h>
    3132
     
    3637
    3738#include <curl/curl.h>
     39
     40namespace WebCore {
    3841
    3942class CurlDownloadManager {
     
    8891
    8992    void init(CurlDownloadListener*, const WebCore::KURL&);
     93    void init(CurlDownloadListener*, ResourceHandle*, const ResourceRequest&, const ResourceResponse&);
     94
    9095    bool start();
    9196    bool cancel();
     
    103108    void closeFile();
    104109    void moveFileToDestination();
     110    void writeDataToFile(const char* data, int size);
     111
     112    void addHeaders(const ResourceRequest&);
    105113
    106114    // Called on download thread.
     
    123131
    124132    CURL* m_curlHandle;
     133    struct curl_slist* m_customHeaders;
    125134    char* m_url;
    126135    String m_tempPath;
     
    137146};
    138147
     148}
     149
    139150#endif
  • trunk/Source/WebKit/win/ChangeLog

    r152908 r152976  
     12013-07-22  peavo@outlook.com  <peavo@outlook.com>
     2
     3        [Curl] Download fails for certain urls.
     4        https://bugs.webkit.org/show_bug.cgi?id=118468
     5
     6        Reviewed by Brent Fulgham.
     7
     8        Initialize download from provided resource handle, request, and response object.
     9
     10        * WebDownload.h: Use WebCore namespace for Curl download class.
     11        * WebDownloadCurl.cpp:
     12        (WebDownload::init): Initialize download from provided resource handle, request, and response object.
     13
    1142013-07-19  peavo@outlook.com  <peavo@outlook.com>
    215
  • trunk/Source/WebKit/win/WebDownload.h

    r151067 r152976  
    4949, public IWebURLAuthenticationChallengeSender
    5050#if USE(CURL)
    51 , public CurlDownloadListener
     51, public WebCore::CurlDownloadListener
    5252#endif
    5353{
     
    142142    RetainPtr<CFURLDownloadRef> m_download;
    143143#elif USE(CURL)
    144     CurlDownload m_download;
     144    WebCore::CurlDownload m_download;
    145145#endif
    146146    COMPtr<IWebMutableURLRequest> m_request;
  • trunk/Source/WebKit/win/WebDownloadCurl.cpp

    r152229 r152976  
    6868    m_delegate = delegate;
    6969
    70     KURL url(ParsedURLString, request.url());
    71 
    72     m_download.init(this, url);
     70    m_download.init(this, handle, request, response);
    7371
    7472    start();
Note: See TracChangeset for help on using the changeset viewer.