Class Downloader

java.lang.Object
org.owasp.dependencycheck.utils.Downloader
public final class Downloader extends Object
A Utility class to centralize download logic like HTTP(S) proxy configuration and proxy- and server credential handling.
Author:
Jeremy Long, Hans Aikema

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    configure(Settings settings)
    Initialize the Downloader from the settings.
    <T> T
    fetchAndHandle(@NonNull URL url, @NonNull org.apache.hc.core5.http.io.HttpClientResponseHandler<T> handler)
    Download a resource from the given URL and have its content handled by the given ResponseHandler.
    <T> T
    fetchAndHandle(@NonNull URL url, @NonNull org.apache.hc.core5.http.io.HttpClientResponseHandler<T> handler, @NonNull List<org.apache.hc.core5.http.Header> hdr, boolean useProxy)
    Download a resource from the given URL and have its content handled by the given ResponseHandler.
    <T> T
    fetchAndHandle(@NonNull org.apache.hc.client5.http.impl.classic.CloseableHttpClient client, @NonNull URL url, @NonNull org.apache.hc.core5.http.io.HttpClientResponseHandler<T> handler, @NonNull List<org.apache.hc.core5.http.Header> hdr)
    Download a resource from the given URL and have its content handled by the given ResponseHandler.
    String
    fetchContent(URL url, boolean useProxy, Charset charset)
    Retrieves a file from a given URL and returns the contents.
    String
    fetchContent(URL url, Charset charset)
    Retrieves a file from a given URL and returns the contents.
    void
    fetchFile(URL url, File outputPath)
    Retrieves a file from a given URL and saves it to the outputPath.
    void
    fetchFile(URL url, File outputPath, boolean useProxy)
    Retrieves a file from a given URL and saves it to the outputPath.
    void
    fetchFile(URL url, File outputPath, boolean useProxy, String userKey, String passwordKey, String tokenKey)
    Retrieves a file from a given URL using an ad-hoc created CredentialsProvider if needed and saves it to the outputPath.
    org.apache.hc.client5.http.impl.classic.CloseableHttpClient
    getHttpClient(boolean useProxy)
    Gets a pre-configured HttpClient.
    static Downloader
    getInstance()
    The singleton instance for downloading file resources.
    org.apache.hc.client5.http.protocol.HttpClientContext
    getPreEmptiveAuthContext()
    Gets a HttpClientContext that supports pre-emptive authentication.
    String
    postBasedFetchContent(URI url, String payload, org.apache.hc.core5.http.ContentType payloadType, List<org.apache.hc.core5.http.Header> hdr)
    Posts a payload to the URL and returns the response as a string.

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • getInstance

      public static Downloader getInstance()
      The singleton instance for downloading file resources.
      Returns:
      The singleton instance managing download credentials and proxy configuration

    • configure

      public void configure(Settings settings) throws InvalidSettingException
      Initialize the Downloader from the settings. Extracts the configured proxy- and credential information from the settings and system properties and caches those for future use by the Downloader.
      Parameters:
      settings - The settings to configure from
      Throws:
      InvalidSettingException - When improper configurations are found.

    • fetchFile

      public void fetchFile(URL url, File outputPath) throws DownloadFailedException, TooManyRequestsException, ResourceNotFoundException, URLConnectionFailureException
      Retrieves a file from a given URL and saves it to the outputPath.
      Parameters:
      url - the URL of the file to download
      outputPath - the path to the save the file to
      Throws:
      DownloadFailedException - is thrown if there is an error downloading the file
      URLConnectionFailureException - is thrown when certificate-chain trust errors occur downloading the file
      TooManyRequestsException - thrown when a 429 is received
      ResourceNotFoundException - thrown when a 404 is received

    • fetchFile

      public void fetchFile(URL url, File outputPath, boolean useProxy) throws DownloadFailedException, TooManyRequestsException, ResourceNotFoundException, URLConnectionFailureException
      Retrieves a file from a given URL and saves it to the outputPath.
      Parameters:
      url - the URL of the file to download
      outputPath - the path to the save the file to
      useProxy - whether to use the configured proxy when downloading files
      Throws:
      DownloadFailedException - is thrown if there is an error downloading the file
      URLConnectionFailureException - is thrown when certificate-chain trust errors occur downloading the file
      TooManyRequestsException - thrown when a 429 is received
      ResourceNotFoundException - thrown when a 404 is received

    • fetchFile

      public void fetchFile(URL url, File outputPath, boolean useProxy, String userKey, String passwordKey, String tokenKey) throws DownloadFailedException, TooManyRequestsException, ResourceNotFoundException, URLConnectionFailureException
      Retrieves a file from a given URL using an ad-hoc created CredentialsProvider if needed and saves it to the outputPath.
      Parameters:
      url - the URL of the file to download
      outputPath - the path to the save the file to
      useProxy - whether to use the configured proxy when downloading files
      userKey - The settings property key for a potentially configured configured Basic-auth username
      passwordKey - The settings property key for a potentially configured configured Basic-auth password
      tokenKey - The settings property key for a potentially configured Bearer-auth token
      Throws:
      DownloadFailedException - is thrown if there is an error downloading the file
      URLConnectionFailureException - is thrown when certificate-chain trust errors occur downloading the file
      TooManyRequestsException - thrown when a 429 is received
      ResourceNotFoundException - thrown when a 404 is received
      Implementation Note:
      This method should only be used in cases where the target host cannot be determined beforehand from settings, so that ad-hoc Credentials needs to be constructed for the target URL when the user/password keys point to configured credentials. The method delegates to fetchFile(URL, File, boolean) when credentials are not configured for the given keys or the resource points to a file.

    • postBasedFetchContent

      public String postBasedFetchContent(URI url, String payload, org.apache.hc.core5.http.ContentType payloadType, List<org.apache.hc.core5.http.Header> hdr) throws DownloadFailedException, TooManyRequestsException, ResourceNotFoundException, URLConnectionFailureException
      Posts a payload to the URL and returns the response as a string.
      Parameters:
      url - the URL to POST to
      payload - the Payload to post
      payloadType - the string describing the payload's mime-type
      hdr - Additional headers to add to the HTTP request
      Returns:
      the content of the response
      Throws:
      DownloadFailedException - is thrown if there is an error downloading the file
      URLConnectionFailureException - is thrown when certificate-chain trust errors occur downloading the file
      TooManyRequestsException - thrown when a 429 is received
      ResourceNotFoundException - thrown when a 404 is received

    • fetchContent

      public String fetchContent(URL url, Charset charset) throws DownloadFailedException, TooManyRequestsException, ResourceNotFoundException
      Retrieves a file from a given URL and returns the contents.
      Parameters:
      url - the URL of the file to download
      charset - The characterset to use to interpret the binary content of the file
      Returns:
      the content of the file
      Throws:
      DownloadFailedException - is thrown if there is an error downloading the file
      TooManyRequestsException - thrown when a 429 is received
      ResourceNotFoundException - thrown when a 404 is received

    • fetchContent

      public String fetchContent(URL url, boolean useProxy, Charset charset) throws DownloadFailedException, TooManyRequestsException, ResourceNotFoundException
      Retrieves a file from a given URL and returns the contents.
      Parameters:
      url - the URL of the file to download
      useProxy - whether to use the configured proxy when downloading files
      charset - The characterset to use to interpret the binary content of the file
      Returns:
      the content of the file
      Throws:
      DownloadFailedException - is thrown if there is an error downloading the file
      TooManyRequestsException - thrown when a 429 is received
      ResourceNotFoundException - thrown when a 404 is received

    • getPreEmptiveAuthContext

      public org.apache.hc.client5.http.protocol.HttpClientContext getPreEmptiveAuthContext()
      Gets a HttpClientContext that supports pre-emptive authentication.
      Returns:
      A HttpClientContext pre-configured with the authentication cache build from the settings.

    • getHttpClient

      public org.apache.hc.client5.http.impl.classic.CloseableHttpClient getHttpClient(boolean useProxy)
      Gets a pre-configured HttpClient. Mainly targeted for use in paged resultset scenarios with multiple roundtrips.
      Parameters:
      useProxy - Whether to use the configuration that includes proxy-settings
      Returns:
      A HttpClient pre-configured with the settings.

    • fetchAndHandle

      public <T> T fetchAndHandle(@NonNull URL url, @NonNull org.apache.hc.core5.http.io.HttpClientResponseHandler<T> handler) throws IOException, TooManyRequestsException, ResourceNotFoundException, URISyntaxException
      Download a resource from the given URL and have its content handled by the given ResponseHandler.
      Type Parameters:
      T - The return-type for the responseHandler
      Parameters:
      url - The url of the resource
      handler - The responsehandler to handle the response
      Returns:
      The response handler result
      Throws:
      IOException - on I/O Exceptions
      TooManyRequestsException - When HTTP status 429 is encountered
      ResourceNotFoundException - When HTTP status 404 is encountered
      URISyntaxException

    • fetchAndHandle

      public <T> T fetchAndHandle(@NonNull URL url, @NonNull org.apache.hc.core5.http.io.HttpClientResponseHandler<T> handler, @NonNull List<org.apache.hc.core5.http.Header> hdr, boolean useProxy) throws IOException, TooManyRequestsException, ResourceNotFoundException, URISyntaxException
      Download a resource from the given URL and have its content handled by the given ResponseHandler.
      Type Parameters:
      T - The return-type for the responseHandler
      Parameters:
      url - The url of the resource
      handler - The responsehandler to handle the response
      hdr - Additional headers to add to the HTTP request
      useProxy - Whether to use the configured proxy for the connection
      Returns:
      The response handler result
      Throws:
      IOException - on I/O Exceptions
      TooManyRequestsException - When HTTP status 429 is encountered
      ResourceNotFoundException - When HTTP status 404 is encountered
      URISyntaxException

    • fetchAndHandle

      public <T> T fetchAndHandle(@NonNull org.apache.hc.client5.http.impl.classic.CloseableHttpClient client, @NonNull URL url, @NonNull org.apache.hc.core5.http.io.HttpClientResponseHandler<T> handler, @NonNull List<org.apache.hc.core5.http.Header> hdr) throws IOException, TooManyRequestsException, ResourceNotFoundException
      Download a resource from the given URL and have its content handled by the given ResponseHandler.
      Type Parameters:
      T - The return-type for the responseHandler
      Parameters:
      client - The HTTP Client to reuse for the request
      url - The url of the resource
      handler - The responsehandler to handle the response
      hdr - Additional headers to add to the HTTP request
      Returns:
      The response handler result
      Throws:
      IOException - on I/O Exceptions
      TooManyRequestsException - When HTTP status 429 is encountered
      ResourceNotFoundException - When HTTP status 404 is encountered