How We Use Middleware

At the heart of Qpoint's universal egress traffic management solution lies our powerful middleware subsystem. Designed for unparalleled flexibility, it enables platform teams and operators to inject custom functions and execute complex traffic manipulations seamlessly. Whether it's out-of-band analytics with Qtap or in-line request modifications with Qproxy, our middleware framework empowers you to tailor egress traffic handling to your unique observability, operational, and security needs.

How it Works

At Qpoint we use Envoy for an integrated proxy for its ability to robustly handle large volumes of traffic with low latency. As a benefit of that we get access to Envoy’s web assembly plugin system which has been carefully architected for performance and security.

This means that extensions, or middleware functions as we call them, are launched in their own virtual machine and are segmented from the rest of the system. Resources are memory mapped into the container, removing the need to copy memory or make RPC calls between systems for data.

Middleware is implemented using WebAssembly for Proxies which implements the Application Binary Interface (ABI) specification. The ABI spec supports L4 and L7 proxies, as well as other environments (for example our Qtap eBPF product). When a middleware function is loaded it is validated to ensure it conforms with the spec before being passed to a virtual machine in each of the Envoy worker threads.

One of the advantages of using the WebAssembly for Proxies interface is that we have the support of the Envoy community as well as other staples like Istio, MOSN, ATS, and OpenResty. This means that the extensions supported by these projects will work with both Qtap and Qproxy.

Another huge advantage is Web Assembly’s support for multiple languages. There are over 20 mainstream languages that compile down to WASM, any of which can implement the spec and create middleware. There are also SDKs with loads of helper functions already implemented for AssemblyScript, C++, Go (TinyGo), and Rust.

Qpoint will automatically handle loading middleware WASM modules from a number of sources:

• From a docker registry
• HTTP (extension .wasm or .tar.gz)
• Locally from disk

Use cases

Middleware takes egress traffic illumination and augmentation to the next level, going beyond traditional, relay-style management like blocking domains, IP addresses, or sources.

Here are a few of the use cases that our customers leverage middleware for today:

Detecting PII Data

The dark web economy of trading leaked data is big business and Data Loss Prevention (DLP) strategies are becoming more mainstream. Qpoint offers a DLP middleware function that checks outgoing payloads for potentially sensitive data, including credit card numbers, email addresses, and social security numbers. It can also report on or (optionally) scrub this data before it leaves your network.

Injecting API keys

Qpoint’s secret injection middleware allows you to actively swap known internal API keys with external (real world) API keys for protection. In addition, you can apply access control to internal keys, so a key used by service A can only make a read request to specific endpoints, while service B can read and write to all endpoints.

Third-party services typically lack access control capabilities that matches your organization security strategy (or have access control at all!). Qpoint’s middleware makes this possible by applying rules based on domain, path, and method.

Detecting & Reporting errors

Knowing when your dependencies are down or degraded is key to keeping customers and services happy. Qpoint’s error detection middleware will surface how third-party API’s are behaving, including degradation, failures, and the sources of those issues. This eliminates the need to build monitoring tooling or to clutter your business logic with robustness checks.

Deep Dive

The Qpoint middleware subsystem leverages an event-driven streaming architecture made accessible through software development kits (SDKs) and a suite of utility packages that simplify common actions.

Streaming API

A streaming API offers significant flexibility in determining middleware behavior and its impact on traffic and performance. For example, a middleware function that needs to perform out-of-band analysis on an HTTP body can capture and reassemble chunks in-flight and analyze them after the call is completed.

If a payload requires modification before reaching its destination, the HTTP call can be paused to enable the body to be modified and reassembled before allowing the request to continue.

Streaming Flow of an HTTP Middleware Function

1. HTTP Request Headers

The entry point of the streaming API for an HTTP request includes the payload headers. These headers can be read, modified, or removed before processing. A property of the function indicates if the stream has ended (e.g., the HTTP call has no body).

2. HTTP Request Body

Once the headers return a Continue flag, the body, if present, can be processed. This function can be called multiple times as chunks arrive. If a middleware function needs to rebuild the body for analysis, it can return a Pause flag until the stream-ending property is true. Otherwise, the Continue flag allows each chunk to pass through.

3. HTTP Request Trailing Headers

Trailing headers are added after the body is processed. These headers often require the body for creation, such as adding a digest for the body. This step is similar to processing Request Headers but occurs after the Body processing call.

4. HTTP Response Headers

Processing HTTP Response headers works similarly to outgoing requests, with an end stream indicating if a body is present.

5. HTTP Response Body

The Response Body follows the same process as the Request Body but applies to the incoming body.

6. HTTP Response Trailing Headers

The Response Trailing Headers call is also similar to the Request Trailing Headers.

7. HTTP Done

The HTTP Done call is the final step in the HTTP call API. It occurs after the complete end-to-end call is finished, making it an ideal place for time-consuming operations such as logging and reporting.

External HTTP Calls

Qpoint provides an implementation for making external HTTP calls, routed through an internal proxy to maintain observability and control over middleware communications. These calls do not pass through the middleware stack.

Middleware functions can make remote calls during any workflow step, including in the root and plugin contexts which will be discussed in a future blog post. Pausing an HTTP request/response mid-flight to make remote HTTP calls can impact latency and should be used sparingly (preferably in the HTTP Done step after the request is complete).

1. Sending HTTP calls

Qpoint offers an internal Envoy cluster for sending remote requests. A utility transaction package simplifies sending traditional Go HTTP request payloads, while core HTTP calls requiring a callback provide more control.

A callback function is required and is triggered upon receiving a response. 

2. Getting HTTP response

When making remote HTTP calls, a callback function is provided. Once invoked, a similar streaming API becomes available for Response Headers, Body, and Trailing Headers.

Conclusion

By leveraging the robust capabilities of Envoy and the flexibility of WebAssembly, Qpoint’s middleware framework offers a powerful toolset for monitoring and managing egress traffic flows across your systems. By utilizing an event-driven, streaming architecture, our middleware subsystem can seamlessly handle complex operations such as data analysis, payload modification, and error detection - all while maintaining high performance and low latency. This ensures that your applications can adapt dynamically to various traffic conditions and security requirements without compromising on speed or efficiency.

Support for multiple programming languages and the extensive community backing of projects like Istio and OpenResty make Qpoint's middleware framework both versatile and future-proof. Whether it's injecting API keys, detecting PII, or monitoring third-party APIs, our middleware subsystem provides a comprehensive framework for improving the observability, reliability, and security of your services. As the landscape of digital interactions continues to evolve, Qpoint's middleware tech stands ready to help you navigate and excel in these changing environments.