Skip to content

Latest commit

 

History

History
234 lines (189 loc) · 6.1 KB

deployment-design-aws.md

File metadata and controls

234 lines (189 loc) · 6.1 KB
Raw

Recommended design for DCL MainNet deployment on AWS

AWS deployment diagram

AWS deployment diagram

Recommended IaC structure and frameworks

  • Ansible - provision of the following node types:
    • Genesis Validator - Validator Node created at the beginning of a network
    • Non-genesis Validator - Validator Node joined a network after a significant time period
    • Private Sentry - Full Node to connect other(external) Validator Nodes (Sentry Node Architecture)
    • Public Sentry - Full Node to connect other(external) Full Nodes
    • Observer - Full Node for serving gRPC / REST / RPC clients
    • Seed - Full Node for sharing IP addresses of Public Sentry Nodes (Seed Node)

Note: Most of the nodes should enable state sync to avoid catching up with a network from scratch. Refer to running-node-in-existing-network.md for details.

  • Terraform - deploy an AWS infrastructure from one or more of the following modules:
    • Validator - Validator node instance
    • Private Sentries - Cluster of Private Sentry node instances
    • Public Sentries - Cluster of Public Sentry node instances with a collocated Seed node
    • Observers - Cluster of Observer node instances
    • Load Balancers - AWS Network Load Balancers for load balancing between Observer clusters

Node specific AWS and DCL configurations

Validator Node

  • Tendermint:

    config.toml file:

    [p2p]
pex = false
persistent_peers = # `Private Sentry` nodes with private IPs
addr_book_strict = false

[consensus]
create_empty_blocks = false
create_empty_blocks_interval = "600s" # 10 mins

    app.toml file:

    [state-sync]
snapshot-interval = "snapshot-interval"
snapshot-keep-recent = "snapshot-keep-recent"

  • AWS:

    • Instance type = EC2 instance
    • Network:
      • Private IPv4 = IPv4 address
      • Public IPv4 = not assigned
    • Security:
      • inbound:
        • allow Tendermint p2p port from Private Sentry Nodes' VPC CIDR
        • allow RPC port from Private Sentry Nodes' VPC CIDR
      • outbound:
        • all

Private Sentry Node

  • Tendermint:

    config.toml file:

    [p2p]
pex = true
persistent_peers = # `Validator` node with private IP + other orgs' validator/sentry nodes with public IPs
private_peer_ids = # `Validator` node id
unconditional_peers = # `Validator` node id
addr_book_strict = false

    app.toml file:

    [state-sync]
snapshot-interval = "snapshot-interval"
snapshot-keep-recent = "snapshot-keep-recent"

  • AWS:

    • Instance type = EC2 instance
    • Network:
      • Private IPv4 = IPv4 address
      • Public IPv4 = Elastic IP
    • Security:
      • inbound:
        • allow Tendermint p2p port for whitelist IPs
        • allow RPC port from Observer Nodes' VPC CIDR
        • allow RPC port from Public Sentry Nodes' VPC CIDR
      • outbound:
        • all

Observer Node

  • Tendermint:

    config.toml file:

    [p2p]
pex = true
persistent_peers = # `Private Sentry` nodes with private IPs
addr_book_strict = false

    app.toml file:

    [api]
enable = true

  • AWS:

    • Instance type = EC2 instance
    • Network:
      • Private IPv4 = IPv4 address
      • Public IPv4 = not assigned
    • Security:
      • inbound:
        • allow gRPC / REST / RPC ports from the same VPC CIDR
      • outbound:
        • all

Public Sentry Node

  • Tendermint:

    config.toml file:

    [p2p]
pex = true
persistent_peers = # `Private Sentry` nodes with private IPs

    app.toml file:

    [state-sync]
snapshot-interval = "snapshot-interval"
snapshot-keep-recent = "snapshot-keep-recent"

  • AWS:

    • Instance type = EC2 instance
    • Network:
      • Private IPv4 = IPv4 address
      • Public IPv4 = Elastic IP
    • Security:
      • inbound:
        • allow Tendermint p2p port from anywhere
        • allow Tendermint RPC port from anywhere
      • outbound:
        • all

Seed Node

  • Tendermint:

    config.toml file:

    [p2p]
pex = true
seed_mode = true
persistent_peers = # `Public Sentry` nodes with public IP

  • AWS:

    • Instance type = EC2 instance
    • Network:
      • Private IPv4 = IPv4 address
      • Public IPv4 = Elastic IP
      • Public DNS = optional
    • Security:
      • inbound:
        • allow Tendermint p2p port from everywhere
      • outbound:
        • all

Load Balancer

  • AWS:

    • Instance type = Elastic Network Load Balancer

    • Availability Zones = [availability zones of observer nodes from the same region]

    • Network:

      • Private IPv4 = IPv4 address
      • Public IPv4 = not assigned
      • Public DNS = assigned by AWS

    • Target groups:

      • gRPC
        • Registered targets = [observer nodes from all availability zones in the same region]
        • Attributes:
          • Preserve client IP addresses = disabled
        • Health checks:
          • protocol = TCP
      • REST
        • Registered targets = [observer nodes from all availability zones in the same region]
        • Attributes:
          • Preserve client IP addresses = disabled
        • Health checks:
          • protocol = TCP
      • RPC
        • Registered targets = [observer nodes from all availability zones the same region]
        • Attributes:
          • Preserve client IP addresses = disabled
        • Health checks:
          • protocol = TCP

    • Listeners:

      • gRPC
        • Protocol = TLS
        • Forward to = gRPC target group
        • Security policy = ELBSecurityPolicy-TLS13-1-2-2021-06
        • Default SSL/TLS certificate = CA signed TLS certificate
      • REST
        • Protocol = TLS
        • Forward to = REST target group
        • Security policy = ELBSecurityPolicy-TLS13-1-2-2021-06
        • Default SSL/TLS certificate = CA signed TLS certificate
      • RPC
        • Protocol = TLS
        • Forward to = RPC target group
        • Security policy = ELBSecurityPolicy-TLS13-1-2-2021-06
        • Default SSL/TLS certificate = CA signed TLS certificate