The Complete Guide to Setting Up Elasticsearch 9.1 and Kibana Locally with Docker Compose

After building search functionality for multiple startups and watching countless developers struggle with Elasticsearch setup, I’ve learned that most tutorials either oversimplify or overcomplicate the process. Here’s the setup that actually works in 2025.

Why Elasticsearch 9.1 Changes Everything

In August 2024, Elasticsearch returned to open source under the AGPL license starting with version 8.16.0. Version 9.1.0 is now fully open source, which means you can finally use it without license concerns for most projects.

I’ve been using Elasticsearch since version 2.x, and let me tell you - the licensing drama is finally over. This is the version you want to learn.

Docker Compose: The Setup That Makes Sense

After years of wrestling with complex Docker run commands and manual configurations, docker-compose is the only way I set up local Elasticsearch environments anymore. It’s platform agnostic, version controlled, and easy to share with your team.

Here’s the configuration I use for all my projects:

The Complete docker-compose.yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
version: '3.8'

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:9.1.0
    container_name: elasticsearch
    environment:
      - node.name=elasticsearch
      - cluster.name=local-cluster
      - discovery.type=single-node
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms1g -Xmx1g"
      - xpack.security.enabled=false
      - xpack.security.enrollment.enabled=false
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - elasticsearch_data:/usr/share/elasticsearch/data
      - ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml:ro
    ports:
      - "9200:9200"
      - "9300:9300"
    networks:
      - elastic
    healthcheck:
      test: ["CMD-SHELL", "curl -f http://localhost:9200/_cluster/health || exit 1"]
      interval: 30s
      timeout: 10s
      retries: 5

  kibana:
    image: docker.elastic.co/kibana/kibana:9.1.0
    container_name: kibana
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
      - SERVER_NAME=kibana
      - SERVER_HOST=0.0.0.0
    ports:
      - "5601:5601"
    networks:
      - elastic
    depends_on:
      elasticsearch:
        condition: service_healthy
    healthcheck:
      test: ["CMD-SHELL", "curl -f http://localhost:5601/api/status || exit 1"]
      interval: 30s
      timeout: 10s
      retries: 5

networks:
  elastic:
    driver: bridge

volumes:
  elasticsearch_data:
    driver: local

Essential Configuration File

Create an elasticsearch.yml file in the same directory:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
cluster.name: "local-cluster"
network.host: 0.0.0.0
http.port: 9200
transport.port: 9300

# Memory settings
indices.memory.index_buffer_size: 10%
indices.memory.min_index_buffer_size: 48mb

# Performance settings for local development
index.number_of_shards: 1
index.number_of_replicas: 0

# Disable security for local development
xpack.security.enabled: false
xpack.monitoring.collection.enabled: false

The 5-Minute Setup Process

Here’s exactly what I run on every new project:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# Create project directory
mkdir elasticsearch-local && cd elasticsearch-local

# Create the docker-compose.yml and elasticsearch.yml files
# (paste the configurations above)

# Start the stack
docker-compose up -d

# Verify Elasticsearch is running
curl http://localhost:9200

# Check cluster health
curl http://localhost:9200/_cluster/health?pretty

In about 2-3 minutes, you’ll see something like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
{
  "cluster_name" : "local-cluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0
}

Memory Configuration That Actually Works

The biggest mistake I see developers make is ignoring memory settings. Elasticsearch will eat your RAM if you let it.

For Development Machines (8GB+ RAM)

Set heap size to 1GB: "ES_JAVA_OPTS=-Xms1g -Xmx1g"
This leaves plenty of room for your IDE, browser, and other apps

For Testing with Larger Datasets

Bump to 2GB: "ES_JAVA_OPTS=-Xms2g -Xmx2g"
Monitor with docker stats elasticsearch

Memory Lock Settings

The bootstrap.memory_lock=true and ulimits configuration prevents Elasticsearch from swapping to disk, which kills performance. I learned this the hard way after debugging slow queries for hours.

Real-World Data Indexing Examples

Let’s index some actual data instead of the usual “hello world” examples. Here’s how I typically test a new setup:

Sample Application Logs

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
# Create an index for application logs
curl -X PUT "localhost:9200/app-logs" -H 'Content-Type: application/json' -d'
{
  "mappings": {
    "properties": {
      "timestamp": {
        "type": "date",
        "format": "yyyy-MM-dd HH:mm:ss"
      },
      "level": {
        "type": "keyword"
      },
      "message": {
        "type": "text",
        "analyzer": "standard"
      },
      "service": {
        "type": "keyword"
      },
      "user_id": {
        "type": "keyword"
      },
      "response_time": {
        "type": "integer"
      }
    }
  }
}
'

# Index some sample log entries
curl -X POST "localhost:9200/app-logs/_doc" -H 'Content-Type: application/json' -d'
{
  "timestamp": "2025-08-04 20:30:15",
  "level": "ERROR",
  "message": "Database connection timeout",
  "service": "user-api",
  "user_id": "user_12345",
  "response_time": 5000
}
'

curl -X POST "localhost:9200/app-logs/_doc" -H 'Content-Type: application/json' -d'
{
  "timestamp": "2025-08-04 20:30:16",
  "level": "INFO",
  "message": "User login successful",
  "service": "auth-service",
  "user_id": "user_67890",
  "response_time": 150
}
'

E-commerce Product Search

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
# Create a product index
curl -X PUT "localhost:9200/products" -H 'Content-Type: application/json' -d'
{
  "mappings": {
    "properties": {
      "name": {
        "type": "text",
        "analyzer": "standard",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "description": {
        "type": "text"
      },
      "price": {
        "type": "float"
      },
      "category": {
        "type": "keyword"
      },
      "tags": {
        "type": "keyword"
      },
      "in_stock": {
        "type": "boolean"
      }
    }
  }
}
'

# Add sample products
curl -X POST "localhost:9200/products/_doc" -H 'Content-Type: application/json' -d'
{
  "name": "MacBook Pro 16-inch",
  "description": "Powerful laptop for developers and creators",
  "price": 2499.00,
  "category": "laptops",
  "tags": ["apple", "laptop", "programming"],
  "in_stock": true
}
'

Kibana: Beyond Basic Dashboards

Open http://localhost:5601 and let’s set up something useful. Skip the sample data and create real visualizations.

Setting Up Index Patterns

Go to Stack Management → Index Patterns
Create pattern for app-logs*
Set timestamp as the time field
Create pattern for products* (no time field needed)

Creating Useful Visualizations

Error Rate Dashboard:

Visualization Type: Line chart
X-axis: Date histogram on timestamp
Y-axis: Count of documents
Filter: level = “ERROR”

This shows you error trends over time - actually useful for monitoring.

Response Time Analysis:

Visualization Type: Histogram
Field: response_time
Interval: 100ms

Perfect for identifying performance bottlenecks.

Search Queries That Matter

Here are the searches I use most often in real applications:

Full-Text Search with Scoring

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
GET /products/_search
{
  "query": {
    "multi_match": {
      "query": "MacBook programming",
      "fields": ["name^2", "description", "tags"],
      "type": "best_fields"
    }
  },
  "highlight": {
    "fields": {
      "name": {},
      "description": {}
    }
  }
}

Complex Filtering and Aggregations

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
GET /app-logs/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "timestamp": {
              "gte": "2025-08-04 20:00:00",
              "lte": "2025-08-04 21:00:00"
            }
          }
        }
      ],
      "filter": [
        {
          "term": {
            "service": "user-api"
          }
        }
      ]
    }
  },
  "aggs": {
    "error_count": {
      "filter": {
        "term": {
          "level": "ERROR"
        }
      }
    },
    "avg_response_time": {
      "avg": {
        "field": "response_time"
      }
    }
  }
}

Troubleshooting Issues Everyone Hits

Container Won’t Start

Error: max virtual memory areas vm.max_map_count [65530] is too low

Solution for Linux:

1
2
sudo sysctl -w vm.max_map_count=262144
echo 'vm.max_map_count=262144' >> /etc/sysctl.conf

Solution for macOS/Windows Docker Desktop: Add this to your docker-compose.yml under the elasticsearch service:

1
2
sysctls:
  - vm.max_map_count=262144

Memory Issues

Symptom: Elasticsearch keeps restarting or becomes unresponsive

Solution: Check Docker memory allocation:

1
docker stats elasticsearch

If memory usage is consistently above 90%, reduce heap size:

1
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"

Port Conflicts

Error: Port 9200 is already in use

Quick fix:

1
2
3
4
# Find what's using the port
lsof -i :9200

# Kill the process or change ports in docker-compose.yml

Kibana Connection Issues

Symptom: Kibana shows “Elasticsearch cluster did not respond”

Check: Verify Elasticsearch health first:

1
curl http://localhost:9200/_cluster/health

If Elasticsearch is healthy but Kibana can’t connect, restart Kibana:

1
docker-compose restart kibana

Performance Tuning for Local Development

Index Settings That Matter

1
2
3
4
5
6
7
# Disable refresh for bulk operations
curl -X PUT "localhost:9200/your-index/_settings" -H 'Content-Type: application/json' -d'
{
  "refresh_interval": "30s",
  "number_of_replicas": 0
}
'

Bulk Indexing for Large Datasets

1
2
3
4
5
6
7
# Use the bulk API for inserting multiple documents
curl -X POST "localhost:9200/_bulk" -H 'Content-Type: application/json' -d'
{"index":{"_index":"products"}}
{"name":"Product 1","price":99.99}
{"index":{"_index":"products"}}
{"name":"Product 2","price":149.99}
'

Production Readiness Checklist

Before you deploy anything based on this setup:

Enable security (set xpack.security.enabled: true)
Configure SSL/TLS certificates
Set up proper authentication
Configure backup strategies
Set up monitoring and alerting
Review memory and storage requirements
Configure log rotation

When Things Go Wrong: My Debugging Process

Check container logs: docker-compose logs elasticsearch
Verify cluster health: curl localhost:9200/_cluster/health
Check resource usage: docker stats
Review configuration: Look for typos in YAML files
Start fresh: docker-compose down -v && docker-compose up -d

What’s Next?

This setup gives you a solid foundation for learning Elasticsearch and Kibana. I use variations of this configuration for:

Application search functionality
Log aggregation and analysis
Real-time analytics dashboards
Performance monitoring

The key is starting simple and adding complexity as you need it. Don’t try to configure everything at once - I’ve seen too many developers get overwhelmed and abandon Elasticsearch entirely.

Want to dive deeper? Start indexing your own application’s data and see what insights you can extract. That’s where Elasticsearch really shines.

Have you set up Elasticsearch locally before? What issues did you run into? I’d love to hear about your experience and help troubleshoot any problems.

Why Elasticsearch 9.1 Changes Everything#

Docker Compose: The Setup That Makes Sense#

The Complete docker-compose.yml#

Essential Configuration File#

The 5-Minute Setup Process#

Memory Configuration That Actually Works#

For Development Machines (8GB+ RAM)#

For Testing with Larger Datasets#

Memory Lock Settings#

Real-World Data Indexing Examples#

Sample Application Logs#

E-commerce Product Search#

Kibana: Beyond Basic Dashboards#

Setting Up Index Patterns#

Creating Useful Visualizations#

Search Queries That Matter#

Full-Text Search with Scoring#

Complex Filtering and Aggregations#

Troubleshooting Issues Everyone Hits#

Container Won’t Start#

Memory Issues#

Port Conflicts#

Kibana Connection Issues#

Performance Tuning for Local Development#

Index Settings That Matter#

Bulk Indexing for Large Datasets#

Production Readiness Checklist#

When Things Go Wrong: My Debugging Process#

What’s Next?#