API Clarity

Basic setup

API Clarity is a console based tool for documenting and managing API by capturing a variety of traffic and generating Open API specs (OAS). The project is available on github.

To get this running you should install k8s, minikube and istio locally with these detailed instructions. There is a demo available on AWS under the EC2 instance api-clarity-demo (sigscidev / Oregon). If you start the instance and ssh onto it there are instructions for getting API clarity running and for generating traffic that will be captured by it. To run on AWS first start the EC2 instance.

To do this go to the AWS EC2 dashboard and from the dashboard - go to Instances (left hand menu) - search for api-clarity-demo - select the checkbox next to the instance - click on Instance state button above the table - then select Start instance

Once the instance state is running ssh onto the machine. With a command like the following:

ssh -i ~/.ssh/nathan-key.pem ubuntu@34.216.73.57

You would need to generate your own ssh key and the IP to use is the Public IPv4 address in the Instance Summary (you need to click on the Instance ID in the Instance table to bring up that page). Once in the home directory you can look at the readme.txt file which gives you the instructions you need to run the API Clarity demo. The file has the contents:

- minikube start
- kubectl get pods (and wait for all pods to be running)
- minikube tunnel
- ./apiclarity.sh (in a new window)
- go to public IP to see the dashboard http://ec2-50-112-2-42.us-west-2.compute.amazonaws.com:9999/ (use the Public IP in Instance Summary)
- ./curl.sh (to generate traffic)
- check the dashboard to see what traffic was generate in the last 5 minutes

Speculator

The speculator is a used by the API clarity engine to transform HTTP traffic specified in JSON format into an OAS spec. You can download the source code from:

Github repo

In order to make it work you would need to make the following code changes in spec.go:

Change the code lines 213-215:

    for path, approvedPathItem := range clonedApprovedSpec.PathItems {
        generatedSpec.Paths.Paths[path] = *approvedPathItem
    }

to

    for path, approvedPathItem := range s.LearningSpec.PathItems {
        generatedSpec.Paths.Paths[path] = *approvedPathItem
    }

Then run

make build

from the root directory.

Create a file telemetry.json with the following contents:

 {
    "destinationAddress": "10.101.253.164:80",
    "destinationNamespace": "namespace",
    "requestID": "abcde123",
    "scheme": "http",
    "sourceAddress": "10.101.253.164",
    "request": {
      "common": {
        "version": "1.0",
        "body": "eyJoZWxsbyI6ICJ3b3JsZCJ9",
        "headers": [{"Key": "Content-type", "Value": "application/json"},
                    {"Key": "Accept", "Value": "application/json"},
                    {"Key": "host", "Value": "www.example.com"}],
        "TruncatedBody": true
      },
      "host": "www.example.com",
      "method": "GET",
      "path": "/product?param=value"
    },
    "response": {
      "common": {
        "version": "1.0",
        "body": "eyJieWUiOiAid29ybGQifQ==",
        "headers": [{"Key": "Content-type", "Value": "application/json"}],
        "TruncatedBody": true
      },
      "statusCode": "200"
    }
  }

After that you can run the command line tool with the following arguments:

./bin/cli learn -t telemetry.json

This should produce an OAS based on the json traffic source.

host: www.example.com:80
info:
  contact:
    email: apiteam@swagger.io
  description: This is a generated Open API Spec
  license:
    name: Apache 2.0
    url: http://www.apache.org/licenses/LICENSE-2.0.html
  termsOfService: http://swagger.io/terms/
  title: Swagger
  version: 1.0.0
paths:
  /product:
    get:
      consumes:
      - application/json
      parameters:
      - in: body
        name: body
        schema:
          properties:
            hello:
              type: string
          type: object
      - in: header
        name: host
        required: true
        type: string
      - in: query
        name: param
        type: string
      produces:
      - application/json
      responses:
        "200":
          description: ""
          schema:
            properties:
              bye:
                type: string
            type: object
        default:
          description: Default Response
          schema:
            properties:
              message:
                type: string
            type: object
swagger: "2.0"