Introducing Dark API Portals. How to cross the gate and access them. Part 1 of 3
We’ll learn how to create & deploy them in Oracle Cloud Infrastructure (OCI), using MuleSoft Anypoint Platform and NetFoundry./OpenZiti.
World of Warcraft has been a very successful gaming franchise for many years. Full disclosure: I’ve never played it — it’s just not my type of game. Despite my indifference, World of Warcraft has been successful enough to captivate many other people around the world.
In 2008 (almost 15 years ago), the book “Beyond the Dark Portal” was published. It is a story that takes place in the World of Warcraft fantasy setting, and as its title suggests, has to do with a Dark Portal, which coincidentally relates to the topic we will cover in this article. Although we will talk about APIs, we will be covering how to prevent malicious forces from attacking them, which is kind of similar to what the book is about!
The fundamental idea of Dark Services is to be invisible to attackers and just be visible to your real consumers and applications.
In those articles, we learned the relevance of publishing APIs as Dark Services in an era when hacks and malicious attacks are happening every single second. We covered how to make your APIs invisible to attackers without opening a single port of your firewall while still allowing public consumption of your APIs.
We still must consider how we can allow potential consumers to search for our Dark API Services.
If the whole idea is to make our services invisible, should we be publishing them through a public API Portal? We would be exposing something which we want to keep as safe as possible. Yet if we do not expose our services publicly, we lose the opportunity for external developers to interact with us.
It sounds like a challenging task, right? On the one hand, we want our APIs to be easily consumed and discoverable; on the other, we want them to be secure and invisible to attackers. We need a way to balance these two opposing initiatives.
The solution is what we’ve called Dark API Portals.
And so, you may be asking: Just what are Dark API Portals?
Let’s go over some of the characteristics of a Dark API Portal:
- A Dark API Portal provides a public user experience, without the need for agents, courtesy of OpenZiti BrowZer; it uses web page authentication to permit access to the private application (the Dark Portal) which has no address on the public internet. Only valid identities can gain access to the Portal. Your Dark API Portal is pretty much invisible to the world but reachable to your registered users.
- A Dark API Portal hosts Dark API Services, which means that an external developer (or an internal one, depending on the use case) will be able to subscribe to Dark API Services just as they could for services hosted in any other public API Portal.
- The developer who wants to access the Dark API Portal will follow the same process as that of any normal API Portal: subscribe to the desired APIs; obtain a client_id/client_secret (or in the case of a provider who requires mTLS, register a certificate); etc. Apart from following the typical API enrollment process, the developer will also need to provide a strong identity to connect to the OpenZiti/NetFoundry overlay network. In other words, to gain access to the API, a developer will not only need to exchange credentials but will also need to have a strong identity issued by NetFoundry.io to use any of its SDKs, such as JAVA, Python, Go, NodeJS, etc.
- From the Dark API Portal, the developers will gain access to the SDKs needed to consume the Dark Services.
You may be wondering now why the lengthy explanation was necessary if we can easily use any type of API portal and control user access to it via username/passwords and Multi-Factor Authentication (MFA). Well, if your interest is to minimize the risk in terms of your public assets, as much as possible, then having it open to the world flies in the face of that. Furthermore, if your APIs need to be as secure as possible, then to have them visible and available to the world is also in conflict with the desired level of security.
None of the current API Management offerings in the market offers a way to publish Dark API Portals and/or Dark Services.
Conventional API portals allow developers to subscribe to APIs; as a result, the API provider creates for them a set of credentials to enable the consumption of the APIs. Conventional API Portals also allow anyone with access to view examples of the APIs, documentation, and Sandboxes which attackers may then use to analyze the usage patterns and consequently try to compromise them.
With Dark API Portals, we can avoid all public information around your APIs being exposed, while at the same time ensuring that they are visible only to your desired consumers. Only your authorized developers will be able to subscribe to the APIs and view all of their details.
Crossing the gate of a Dark API Portal will allow you to see things that no one else can see.
The wardrobe that takes you to Narnia. The Chronicles of Narnia.
Entering a Dark API Portal will allow developers to browse, subscribe, and consume APIs that no one else can.
In the Chronicles of Narnia, crossing through a portal into Narnia is done by way of regular, well-known methods: in our case, we cross through a portal via a web browser. The user/consumer/developer will use a common browser (e.g. Google Chrome) to gain access to the Dark API Portal by executing the steps of the following flow:
As you can see in the above flow, OpenZiti BrowZer integrates with an Identity Provider (IdP), which will coordinate the authentication process for users who want access to the Dark API Portal. Once the user has successfully authenticated, control is returned to OpenZiti BrowZer which then uses the user identity provided by the IdP to obtain authorization for access to the NetFoundry.io overlay network. Depending on the authorization granted, the user identity may or may not be able to access the application (using the App WAN concept); if authorization has been granted, then the content is served.
The following is so important, it bears repeating:
The Dark API Portal is invisible to everyone except valid users. On the public internet, API Portals are visible to even invalid users who may try to attack and compromise them.
So how do we build/create a Dark API Portal?
Property of Netflix. DARK
In this series of articles we are going to use the following:
- MuleSoft as the API Manager to expose and manage the APIs
- MuleSoft Admin APIs to expose all the functionality necessary to create a custom API Portal to allow the listing of APIs with their contracts and also to allow external subscription to the APIs
- NetFoundry.io and its APIs to interact with the Dark API Portal to create a new endpoint and attach it to an App WAN
- Netfoundry.io/OpenZiti BrowZer to allow users to gain access to the Dark API Portal
- A React.js web application, which is the Dark API Portal, running on a fully private compute instance deployed in a private subnet within Oracle Cloud Infrastructure
- A Mule Runtime server which also runs on a fully private compute instance deployed in a private subnet within Oracle Cloud Infrastructure
A NetFoundry.io Edge Router deployed on a private subnet within Oracle Cloud Infrastructure - Postman Flows to explain the steps
(Notice that we are not going to open a single port in the Firewall to make both the Dark API Portal and its Dark Services available from an external network).
The following diagram describes how the different elements are distributed:
Oracle Cloud Infrastructure
As you can see, all is private within the VCN; the consumers are not connected directly to the services, nor do they even know where they are.
Our Dark API Portal is sitting on that compute instance labeled as My Dark Portal within a Private Subnet. Also, within that very same Private Subnet, we have a Mule Runtime server which is completely private and reachable only through that subnet.
We also have our NetFoundry.io Edge Router sitting on a private subnet. There is nothing publicly available, nor do we have reverse proxies, bastions, or VPNs to allow access from outside of that VCN. The Security Lists have open ports just within the private subnets — nothing from the internet or other networks.
Alright, now we will explain how to use MuleSoft and NetFoundry.io to create the Dark API Portal.
Max & Ziggy getting together
Since the out-of-the-box API Portals are not prepared to become Dark, we will leverage the advanced functionality of their admin APIs to create them instead.
We need to create a web application that can interact with the API Manager (i.e. the MuleSoft Anypoint API Manager) to list the available APIs of a given organization and allow developers to subscribe to them. MuleSoft offers several resources to do so through its Access Management API. We just need a subset of the resources offered by the Access Management API. Specifically, we need the following:
- Authentication resource
- Organization resources to retrieve the organization ID
- The environment resource to retrieve the environment ID
- The API resource to get the list of APIs of a specific environment and a specific organization
- The API resource to get the details of a specific API
- The Application resource will be used to subscribe to the desired API to either create a new application or reuse an existing one
- A Tier resource, to get the different tiers of an API (e.g. silver, gold, platinum) which can be used to limit the number of calls a developer can make to an API
- A Contracts resource to subscribe to a created or existing application (as mentioned in point #6 of this list), to register with the desired API whose details were retrieved in step #5
Take a look at the following Postman Flow:
Postman Flow to create an Endpoint and associate it with an App WAN in NetFoundry
The Postman Flow and Collection are available here for your convenience.
At last! We have all the pieces in place now to create our Dark API Portal. In the next article, we will dive deep into how to use all those resources to make it happen. Stay tuned!
Summary:
We introduced the concept of Dark API Portals and how to implement them using MuleSoft Anypoint Platform and NetFoundry/OpenZiti
We described the relevance of creating them
We described the elements needed to make it happen
In the next two articles, you will learn in detail how to create them