Kill Bill monitoring with Datadog

Kill Bill monitoring with Datadog

The Kill Bill team loves data and measuring things. As part of our performance work last year, we did a pass at cleaning up all of our metrics. As a result, we now export over 600 metrics all over the stack, from latency of low-level database queries to overall rates of HTTP requests. These can be easily integrated with InfluxDB and Grafana: that’s how Groupon monitors its payment infrastructure. Maintaining a monitoring system is no easy task however, especially at scale. It’s no surprise that even companies like Stripe are outsourcing (some of) this work to Datadog. In this post, I’ll show you how to export Kill Bill metrics with Datadog’s standard agent, and some of the metrics you want to keep an eye on in production. I assume you already have Kill Bill running using our Docker image and that you have a Datadog API key. The first step is to enable JMX, by adding the following System Properties to your kpm.yml: ~ > docker exec -ti killbill /bin/bash [email protected]:/var/lib/tomcat7$ export TERM=xterm [email protected]:/var/lib/tomcat7$ vi /etc/killbill/kpm.yml # Add the following to the jvm: line # -Dcom.sun.management.jmxremote=true # -Dcom.sun.management.jmxremote.authenticate=false # -Dcom.sun.management.jmxremote.port=8000 # -Dcom.sun.management.jmxremote.ssl=false [email protected]:/var/lib/tomcat7$ exit ~ > docker restart killbill Then, install the Ubuntu Datadog agent (when prompted, the sudo password is tomcat7): ~ > docker exec -ti killbill /bin/bash [email protected]:/var/lib/tomcat7$ cd /var/tmp [email protected]:/var/lib/tomcat7$ DD_API_KEY=REPLACE_ME bash -c "$(curl -L https://raw.githubusercontent.com/DataDog/dd-agent/master/packaging/datadog-agent/source/install_agent.sh)" [email protected]:/var/lib/tomcat7$ sudo /etc/init.d/datadog-agent start Note that the final installation step, when the agent tries to register with your account, may fail. Starting the agent manually afterwards fixes the issue. Go now to you Datadog account and enable the JMX integration. The final step is...
ChartMogul integration

ChartMogul integration

Running a successful subscription business requires careful monitoring of a variety of metrics: MRR, Churn, LTV, CAC, etc. It’s no surprise that our first plugin was the Analytics plugin, which computes in realtime these metrics and provides tools to build your own dashboards. Understanding, tweaking and validating these KPIs for your own business requires careful thinking and is typically owned by an experienced Business Intelligence team working closely with the executive team on a daily basis. For those who simply can’t afford the overhead, ChartMogul comes to the rescue. Their SaaS platform builds these metrics for you and lets you explore your data through cohorts, custom segments and geo-mapping tools. We’re pleased to announce today a new Kill Bill plugin for ChartMogul. This integration is in fact very similar to the Analytics plugin, i.e. it listens in real-time to events that Kill Bill produces, and mirrors the data in ChartMogul:       The plugin is available in beta at https://github.com/killbill/killbill-chartmogul-plugin. For help or feedback, feel free to reach-out on the...
Kill Bill on Microsoft Azure

Kill Bill on Microsoft Azure

The open-source Kill Bill platform provides advanced subscription management as well as payment APIs. Businesses of any size, from the 2-person startup to public companies, rely on it for their e-commerce recurring billing and shopping cart payment needs. Lots of SaaS companies have similar offerings. Choosing the right partner from the get go is critical however, as migrations are error-prone and difficult. If you are a young startup, you probably want something simple, cheap and low-maintenance, which can adapt to your needs as you grow. Kill Bill combined with Microsoft Azure BizSpark for startups gives you just that, which probably explains the uptick of requests we’ve had recently regarding integration with Microsoft Azure. Thanks to our Docker images, this is quite easy – this tutorial will show you how to deploy Kill Bill integrated with Stripe in a few commands. Pre-requisites Create a Microsoft Azure account Write down your subscription id: log-in to https://portal.azure.com, click on the account dropdown, then “My permissions” Install the latest version of docker-machine VM setup Create the VM using docker-machine: docker-machine create -d azure \ --azure-ssh-user ops \ --azure-subscription-id AZURE_SUBSCRIPTION_ID \ --azure-open-port 8080 \ --azure-open-port 3000 \ azure You should see it running at https://portal.azure.com/#blade/HubsExtension/BrowseAllResourcesBlade. Write down the IP of the machine by running docker-machine ip azure Configure your shell for the rest of the tutorial: eval $(docker-machine env azure) Database setup For this tutorial, we will simply run a MySQL database in a Docker container on the same host: docker run -tid \ --name db \ -p 3306:3306 \ -e MYSQL_ROOT_PASSWORD=root \ -e MYSQL_DATABASE=killbill \ mariadb Modify the database to make sure it...
Understanding payment failures

Understanding payment failures

A typical integration with a payment gateway such as Stripe looks like this: In the error handling block, you may decide to email the customer, block his access to your website, etc. Hopefully, you also keep track of these errors and generate a daily report on how many transactions fail: after all, these directly impact your bottom line. Good enough? Unfortunately, as we will see below, this approach is often too simplistic. In this particular example, the exception caught is only one of several possibilities. Looking at Stripe’s documentation specifically, CardError is thrown only for 402 – Request Failed. Other exceptions can occur, such as RateLimitError, when 429 – Too Many Requests is returned: in this case, you went over your API quota and you will want such requests to be re-submitted later. Let’s now assume we fix our code above to look like: Errors like InvalidRequestError, AuthenticationError or APIError should be closely monitored after each deployment, as they may be triggered by a recent code or configuration change. What about the black box of CardError? Let’s take a look at how we can categorize further such payment failures. On one hand, some transactions may be blocked by your payment gateway, depending on your configuration settings. Braintree for instance refers to them as Gateway Rejections. A typical example is CVV rejections: when triggering an authorization, the issuing bank will tell the gateway if the supplied CVV matches. If it doesn’t, most gateways can be configured to automatically void the transaction to prevent fraud. Another typical scenario is when your provider offers fraud detection tools: if it determines that the...
Postman Collections for Kill Bill

Postman Collections for Kill Bill

We consider Kill Bill as a platform first, upon which you can build your billing and payment infrastructure. As such, we are working hard to make sure (almost) every aspect of the system is accessible through APIs (subscriptions, invoices and payments management, but also system configuration, plugins management, etc.). As we’re approaching 200 Kill Bill APIs, interacting with the system can be intimidating. I personally have entire folders full of cURL recipes, for Kill Bill itself but also for our various plugins. We publish our documentation using Swagger (available at runtime at http://127.0.0.1:8080/api.html), which is incredibly useful for discovering APIs and one-off requests, less so for keeping a library of snippets. Enter Postman, a free Chrome and Mac App which helps you construct, send and store HTTP requests. Once you have it installed, just import the Kill Bill Postman Collection. Select a request template, modify the query parameters and/or JSON body and click “Send”. Voilà! Postman sends the request to Kill Bill and automatically pretty-prints the response. You can even generate the equivalent cURL command (useful to share it on a bug report). You can also create your own private Collections, for example one for common requests you make in your development environment (trigger a payment, restart a plugin, etc.) and one for requests to help debugging your production environment (e.g. retrieve details for a specific account or payment). I’ve just started using it, mainly as a fancy repository of cURL snippets, but it offers lots of advanced features like requests scripting, a test runner and team collaboration tools. Go give it a...