curl - macOS

Overview

curl is a powerful command-line tool used to transfer data to or from a server using various protocols such as HTTP, HTTPS, FTP, FTPS, SCP, SFTP, and more. Primarily, it is used to download files or web pages from the internet. Its versatility makes it an essential tool for testing and interacting with APIs, automating file downloads, and web scraping.

Syntax

The basic syntax of curl is:

curl [options] [URL...]
  • options: These are flags that modify the behavior of curl.
  • URL: The URL to which curl makes the request must be enclosed in quotation marks if it contains special characters.

Options/Flags

  • -o, --output <file>: Write output to <file> instead of stdout.
  • -O, --remote-name: Save the download file with the remote filename.
  • -L, --location: Follow redirects.
  • -H, --header <header>: Pass custom header(s) to the server.
  • -d, --data <data>: Sends the specified data in a POST request to the HTTP server.
  • --data-urlencode <data>: URL-encodes the given data and appends it to the URL.
  • -u, --user <user:password>: Pass user and password for server authentication.
  • -I, --head: Fetch the headers only.
  • -X, --request <command>: Specify the request command to use.
  • -s, --silent: Silent or quiet mode. Don’t show progress meter or error messages.
  • -v, --verbose: Make the operation more talkative.
  • --limit-rate <rate>: Limit the data transfer rate specified in bytes per second.
  • -A, --user-agent <agent string>: Send User-Agent <agent string> to the server.

Examples

  1. Downloading a File

    curl -O http://example.com/file.tar.gz

  2. Using Custom Headers

    curl -H "X-My-Header: 123" http://example.com

  3. POST Data

    curl -d "login=username&password=secret" https://example.com/login

  4. Save Output to File

    curl -o mypage.html http://example.com

  5. Verbose Mode

    curl -v https://example.com

Common Issues

  • Error: Could not resolve host: Ensure the URL is correct or check your internet connection.
  • Permission Denied: When writing to a location where you don’t have write permission. Use sudo or change the target directory.
  • SSL Certificate Problem: Use -k or --insecure to ignore certificate validation, but only on trusted sites.

Integration

curl can be combined with other tools like jq for processing JSON data:

curl -s https://api.example.com/data | jq '.'

Or use it in scripts to check connectivity:

if curl -s --head --request GET https://example.com | grep "200 OK" > /dev/null; then
   echo "Website is UP."
else
   echo "Website is DOWN."
fi
  • wget: Another command-line tool to download files from the internet.
  • httpie: A modern, user-friendly command-line HTTP client.

For further reading and more detailed information, refer to the official curl documentation.