Connect to AWS OpenSearch in VPC Private Subnet using EC2 Instance Connect Endpoint (Local/Laptop)

Table Of Contents

• Introduction
• Upgrade AWS CLI
• Create EC2 Instance Connect Endpoint
• Get Private IP Address of AWS OpenSearch Service
• OpenSearch Security Groups
• OpenTunnel
• Connect to AWS OpenSearch Cluster

Introduction

EC2 Instance Connect Endpoint allows you to connect to an instance without requiring the instance to have a public IPv4 address.

Upgrade AWS CLI

You need a minimum of AWS CLI v2.12+ installed in your local to be able to use EC2 Instance Connect Endpoint Service.

You can find the upgrade instructions for your CLI installation here - https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html

Create EC2 Instance Connect Endpoint

Next, we create an EC2 Instance Connect Endpoint in your VPC which have the resources you would like to connect to e.g. AWS OpenSearch Domain

EC2 Instance Connect Endpoint has following limitations:

• 5 EC2 Instance Connect Endpoints per AWS account

• 1 EC2 Instance Connect Endpoint per VPC

• 1 EC2 Instance Connect Endpoint per subnet

Also, you will be charged for cross-AZ traffic if your EC2 Instance Endpoint connects to a resource running in another AZ.

Finally, AWS will throttle any high volume data transfers done through this service.

You can create the EC2 Instance Connect Endpoint through the VPC service VPC -> Endpoints -> Create endpoint

Provide a VPC, Subnet & a security group to be used for the endpoint service.

For example, you can choose your VPC default security group here or create a custom security group.

Get Private IP Address of AWS OpenSearch Service

Get the VPC URL of the OpenSearch domain and do nslookup on the same to get the private IP addresses of the cluster.

e.g.

$ nslookup vpc-test-dgseryegg3hnujihusfg.us-east-1.es.amazonaws.com
Server: 10.12.3.10
Address: 10.12.3.10#12

Non-authoritative answer:
Name: vpc-test-dgseryegg3hnujihusfg.us-east-1.es.amazonaws.com
Address: 10.1.12.2
Name: vpc-test-dgseryegg3hnujihusfg.us-east-1.es.amazonaws.com
Address: 10.1.14.3

We can use any of the IP’s above e.g. 10.1.12.2 to open a tunnel to the OpenSearch cluster.

OpenSearch Security Groups

You need to ensure your OpenSearch security group allows traffic from your EC2 Instance Connect Endpoint.

This can be achieved in the following ways:

[1] Add the EC2 Instance Connect Endpoint security group as an Inbound rule to your OpenSearch security group.

[2] Add VPC CIDR as an inbound rule to the OpenSearch security group.

OpenTunnel

Run below command to open a tunnel to your OpenSearch domain:

aws ec2-instance-connect open-tunnel --private-ip-address 10.1.12.2 --instance-connect-endpoint-id <your-instance-connect-endpoint-id> --remote-port 443 --local-port 9200

Connect to AWS OpenSearch Cluster

You will see below output once you ran OpenTunnel command:

Listening for connections on port 9200

You can now connect to the OpenSearch domain from your local using your tool of your choice e.g. Postman.

Open Postman and ensure the following:

[1] Choose GET as the request type.

[2] Enter list indices API in the URL https://localhost:9200/_cat/indices - you need to use https here otherwise you will get 400 The plain HTTP request was sent to HTTPS port error

[3] Turn off SSL certificate verification.

[4] If your domain has fine grained access control enabled, then you will have to pass the credentials for the API requests. So, set Basic Auth in the request and configure Username & Password.

Execute the request which will list the indices in your domain.