Develop webhooks locally using Cloudflared Tunnel
This blog post was written for Twilio and originally published at the Twilio blog.
Webhooks are a way to be notified by an external service when an event has occurred. Instead of you sending an HTTP request to that service, the service sends an HTTP request to your public web service. This way, you can respond to the event in real-time as it happens. Webhooks are also a common way to integrate with Twilio's products. For example, when your Twilio phone number receives a text message or phone call, Twilio sends an HTTP request to your service with the details. Your service then responds with instructions that indicate how Twilio should respond to the event. Here's a diagram of what this exchange looks like:
One common challenge with webhooks is that they can only call web services that are publicly available on the internet, but when you are initially developing software, you are typically doing so on your own local machine, which by default is not reachable via the internet. So, how do you develop and test webhooks on your local machine? The answer: tunnels. In this post, you'll learn how to use the Cloudflare Tunnels service to make your local web server publicly accessible on the internet. Then you'll learn how to use the tunnel service to respond to a Twilio webhook to respond to a phone call.
Local tunnels #
You can create a tunnel between your local machine and a tunnel service to expose local ports to the public internet. There are many tools that can help you do this, and today you're going to learn about Cloudflare Tunnel or cloudflared (formerly known as Argo Tunnel).
Prerequisites #
You will need the following things to follow along:
- A Linux, macOS, or Windows machine
- Your development stack of choice (Node.js, PHP, .NET, Java, Python, etc.)
- Optional: A Cloudflare account with a website that uses Cloudflare DNS as their name server
- Optional: A Twilio Account (If you register here, you'll receive $10 in Twilio credit when you upgrade to a paid account!)
Set up a local web server #
You can tunnel a local port at any time, but you won't see any results if nothing is listening to that port on your machine. In this section, you'll set up a local web server for serving static files.
Usually, the response for a webhook is created programmatically, but for the purposes of this demo, you'll use static files.
Open your preferred shell, create a folder, and navigate to it:
mkdir MyStaticSite
cd MyStaticSite
Create a new file index.html in the MyStaticSite folder with the following content:
<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Hello World!</title> </head> <body> Hello World! </body> </html>
You'll need to run a static web server to serve this HTML file. There are many static servers for every development stack out there. Pick your preferred stack below and follow the instructions to serve the current folder at http://localhost:8080/.
Node.js: Run the following command to execute the http-server NPM package and run it:
npx http-server . --port 8080
PHP: You can use the built-in PHP web server by running the following command:
php -S localhost:8080
.NET: You can use the dotnet-serve tool to run a static web server. Install the tool as a global .NET tool:
dotnet tool install --global dotnet-serve
Run the tool to serve the current directory:
dotnet serve --port 8080
Python 2:
python -m SimpleHTTPServer 8080
Python 3:
python -m http.server 8080
Check out this list of static web server commands if your preferred programming language is not listed above.
Leave the static server running and open a new shell for the upcoming commands.
Tunnel your local web server using Cloudflare Tunnel #
You can use the Cloudflare Tunnel client to tunnel your local ports to the public internet.
You'll need to use the cloudflared CLI tool for this. Download the cloudflared command line tool here.
The quickest way to start tunneling your local machine is to run this command:
cloudflared tunnel --url http://localhost:8080
This command will tunnel http://localhost:8080 to a randomly generated trycloudflare.com subdomain. For example, when I ran this command, the publicly available address was https://colleagues-ga-eternal-recruitment.trycloudflare.com.
Here's what the output should look like:
Thank you for trying Cloudflare Tunnel. Doing so, without a Cloudflare account, is a quick way to experiment and try it out. However, be aware that these account-less Tunnels have no uptime guarantee. If you intend to use Tunnels in production you should use a pre-created named tunnel by following: https://developers.cloudflare.com/cloudflare-one/connections/connect-apps Requesting new quick Tunnel on trycloudflare.com... +--------------------------------------------------------------------------------------------+ | Your quick Tunnel has been created! Visit it at (it may take some time to be reachable): | | https://colleagues-ga-eternal-recruitment.trycloudflare.com | +--------------------------------------------------------------------------------------------+ ... Much more!
You can find your randomly generated address inside the box printed to the console. Go to the address using a web browser. You'll see the HTML file that is served from your local web server is now publicly served at the address Cloudflare gave you.
You can use this public address and configure it as your webhook address with the external services you want to integrate with.
Use a persistent address for your Cloudflare Tunnel #
When you stop the tunnel command, the tunnel will be destroyed. When you run the tunnel command again, a new tunnel with a new random URL will be created. This also means you have to keep updating the webhook URL at your external service, which can be a hassle. Some of these tunnel services do offer persistent URLs as part of their paid plan. But with Cloudflare Tunnels, you can have persistent URLs for free if you use Cloudflare DNS. Instead of using a randomly generated URL, you can create a subdomain in Cloudflare DNS and use that as your persistent tunnel URL.
If you choose to use a persistent URL, you'll need to
- Create a Cloudflare account and add your website
- Verify you own your domain name
- Use Cloudflare DNS as your domain name servers
Feel free to skip this section If you don't already use Cloudflare DNS for your website and you don't want to. You can keep using the randomly generated address for the next section.
Log into Cloudflare using the following command:
cloudflared tunnel login
This will open Cloudflare's login page. After logging in, select the domain you want to use for tunneling.
When prompted whether you want to authorize this domain, click Authorize.
Switch back to your shell, and you should see "You have successfully logged in." in the output.
Create a new tunnel by running this command:
cloudflared tunnel create myfirsttunnel
You created a tunnel with name myfirsttunnel. You can list all your tunnels using the following command:
cloudflared tunnel list
Run the above command and take note of the ID of your tunnel.
By now, the CLI tool has created a folder to store its configuration. On Windows, this is located at %USERPROFILE%\.cloudflared. On Linux and macOS, you can find this folder at ~/.cloudflared. Create a new file myfirsttunnel.yaml in the .cloudflared folder and paste in the following content:
url: http://localhost:8080
tunnel: <Tunnel-UUID>
credentials-file: /root/.cloudflared/<Tunnel-UUID>.json
This configuration file will configure the local tunnel to listen to http://localhost:8080.Replace <Tunnel-UUID> with the tunnel ID you took note of earlier.
Run the following command to specify the subdomain you want to use for your public tunnel URL:
cloudflared tunnel route dns myfirsttunnel mysubdomain
In this command, myfirsttunnel specifies the name of the tunnel, and mysubdomain specifies the subdomain you want to use.
Now that everything is configured, you can run your tunnel using this command:
cloudflared tunnel run myfirsttunnel
Your new tunnel URL should be: https://<your-subdomain>.<your-domain>.<your-tld>. For example, mine would be https://mysubdomain.swimburger.net.
Open your new tunnel URL in a browser to verify that the same HTML file is being returned.
Use your tunnel with Twilio webhooks #
You can also use these tunnels to respond to Twilio webhooks. In this section, you'll learn how to respond to a phone call using a static XML file.
Create a new file call.xml in the MyStaticSite folder:
<?xml version="1.0" encoding="UTF-8"?> <Response> <Say voice="alice">Twilio received these instructions from your local machine via Cloudflare Tunnel.</Say> <Pause length="1" /> <Say voice="alice">Great job!</Say> </Response>
This XML file uses TwiML, also known as the Twilio Markup Language. Using TwiML, you can provide instructions on how Twilio should respond to an incoming call or SMS. The TwiML above uses two TwiML Voice verbs: Say and Pause. Say converts text to speech, and the voice attribute specifies which voice to use. Pause will wait the specified amount of time, which is one second in this case, and then move on to the next verb.
Make sure your static web server is still running, and run the Cloudflare tunnel using cloudflared.
Browse to the XML file using the publicly tunneled URL: [tunnel-url]/call.xml
The XML should be returned to your browser. Take note of this URL.
If you don't already have a Twilio account, you can get a free one. (If you register here, you'll receive $10 in Twilio credit when you upgrade to a paid account!)
Next, you'll need a Twilio phone number. Go and buy a new phone number from Twilio. The cost of the phone number will be applied to your free promotional credit.
Use the left side navigation to navigate to Phone Numbers > Manage > Active numbers. Click on your Twilio phone number to navigate to its settings.
Find the A CALL COMES IN fields under the Voice & Fax section. Change the text field to your public tunnel URL with the /call.xml path, and change the HTTP verb to HTTP GET.
Click Save to submit the form.
To verify the call is working as expected, call your Twilio phone number using your personal phone. You should hear "Twilio received these instructions from your local machine via Cloudflare Tunnel. Great job!".
Using a static file is sufficient to demonstrate local tunneling, but you can also use the same techniques in combination with any web server technology to process and respond to the HTTP request.
Developing webhooks locally with Cloudflared tunnel #
Webhooks are a great way to be notified by an external service when an event has occurred. The external service makes an HTTP request to your public web service and provides the details of the event in the body of the request. You can use local tunneling to make your local port accessible on the internet using a tunnel service like Cloudflare Tunnels. Using Cloudflare Tunnels, you can tunnel your local port to a random public URL provided by Cloudflare. If you use Cloudflare DNS, you can also configure a persistent tunnel URL for free.
Many Twilio products have webhook capabilities. Learn how you can infuse communication technology into your application in the Twilio Docs, and learn how to get started quickly with Twilio's SDKs.
