self-healing open source data connector 🍯

superglue is a self-healing open source data connector. You can deploy it as a proxy between you and any complex / legacy APIs and always get the data that you want in the format you expect.

Here's how it works: You define your desired data schema and provide basic instructions about an API endpoint (like "get all issues from jira"). Superglue then does the following:

• Automatically generates the API configuration by analyzing API docs.
• Handles pagination, authentication, and error retries.
• Transforms response data into the exact schema you want using JSONata expressions.
• Validates that all data coming through follows that schema, and fixes transformations when they break.

If you’re spending a lot of time writing code connecting to weird APIs, fumbling with custom fields in foreign language ERPs, mapping JSONs, extracting data from compressed CSVs sitting on FTP servers, and making sure your integrations don’t break when something unexpected comes through, superglue might be for you.

flowchart LR
    subgraph Input[data sources]
        A1[APIs]
        A2[files]
        A3[legacy systems]
    end

    subgraph Process[data transformation]
        T1[superglue engine]
    end

    subgraph Output[destination]
        D1[your system]
    end

    Input --> Process
    Process --> Output

    %% Styling
    classDef sourceStyle fill:#f9f,stroke:#333,stroke-width:2px
    classDef processStyle fill:#bbf,stroke:#333,stroke-width:2px
    classDef outputStyle fill:#bfb,stroke:#333,stroke-width:2px

    class Input sourceStyle
    class Process processStyle
    class Output outputStyle

quick start

hosted version

1. Sign up for early access to the hosted version of superglue at superglue.cloud

2. Install the superglue js/ts client:

npm install @superglue/client

3. Configure your first api call:

import { SuperglueClient } from "@superglue/client";

const superglue = new SuperglueClient({
  apiKey: "************"
});

const config = {
  urlHost: "https://futuramaapi.com",
  urlPath: "/graphql",
  instruction: "get all characters from the show",
  responseSchema: {
    type: "object",
    properties: {
      characters: {
        type: "array",  
        items: {
          type: "object",
          properties: {
            name: { type: "string" },
            species: { type: "string", description: "lowercased" }
          }
        }
      }
    }
  }
};

const result = await superglue.call({endpoint: config});
console.log(JSON.stringify(result.data, null, 2));

/*
output:
{
  "characters": [
    {
      "name": "Phillip J. Fry",
      "species": "human"
    },
    ...
  ]
}
*/

self-hosted version

Run your own instance of superglue using Docker:

1. Pull the Docker image:

docker pull superglueai/superglue

2. Create a .env by copying the .env.example file at the root

3. Start the server:

docker run -d \
  --name superglue \
  --env-file .env \
  -p 3000:3000 \
  -p 3001:3001 \
  superglueai/superglue

4. Verify the installation:

curl http://localhost:3000/health
> OK

# or open http://localhost:3000/?token=your-auth-token

5. Open the dashboard to create your first configuration:

http://localhost:3001/

6. run your first call:

npm install @superglue/client

import { SuperglueClient } from "@superglue/client";

const superglue = new SuperglueClient({
  endpoint: "http://localhost:3000",
  apiKey: "your-auth-token"
});

// either via config object
const config = {
  urlHost: "https://futuramaapi.com",
  urlPath: "/graphql",
  instruction: "get all characters from the show",
};

const result = await superglue.call({endpoint: config});

// or via the api id if you have already created the endpoint
const result2 = await superglue.call({id: "futurama-api"});

console.log(JSON.stringify(result.data, null, 2));

key features

• LLM-Powered Data Mapping: Automatically generate data transformations using large language models
• API Proxy: Intercept and transform API responses in real-time with minimal added latency
• File Processing: Handle various file formats (CSV, JSON, XML) with automatic decompression
• Schema Validation: Ensure data compliance with your specified schemas
• Flexible Authentication: Support for various auth methods including header auth, api keys, oauth, and more
• Smart Pagination: Handle different pagination styles automatically
• Caching & Retry Logic: Built-in caching and configurable retry strategies

📖 Documentation

For detailed documentation, visit docs.superglue.cloud.

🤝 contributing

We love contributions! Feel free to open issues for bugs or feature requests.

license

superglue is GPL licensed. The superglue client SDKs are MIT licensed. See LICENSE for details.

🙋‍♂️ support

• 💬 Discord: Join our community
• 🐛 Issues: GitHub Issues