Skip to main content

Kubernetes Clusters

This page is a draft

We have several Kubernetes clusters at the Mesh:

    dev3 (SN3)
      VM: meshdev3mgr0 on nycmesh-713-r640-02 Manager IP: 10.70.90.146 Node IPs: 10.70.90.153, 10.70.90.154, 10.70.90.155, 10.70.90.157, 10.70.90.161 User: debian (must log in using console, no keyboard-interactive SSH) prod1 (deprecated,SN3) runs
        VM: meshprod1mgr0 on hardwarenycmesh-713-r640-02
        weManager wantIP: to10.70.90.152 divestNode toIPs: Zach10.70.90.147, for10.70.90.143, routing)
        10.70.90.144

        User: debian (must log in using console, no keyboard-interactive SSH) prod2 (mainSN10)
        production
        instance)
        VM:

        dev3meshprod2mgr0 on nycmesh-10-r630-01

        Manager IP: 10.70.100.47 Node IPs: 10.70.100.48, 10.70.100.49, 10.70.100.50 User: debian (pre-production)
        must

        log in using console, no keyboard-interactive SSH)

        gamma1 ("staging" environment to test changes in a prod-like environment) (turned off)

        We use them to run software services, and ideally abstract as much faffing about away from someone who wants to write a web service as possible.

        If you'd like to work with one, just ask around in #mesh-services. James or Willard can get you sorted.

        Standing up a Cluster

        To stand up a cluster, follow these steps.

        Find a range of IP Address space in the datacenter you want to host in. Reserve it in the IPRanges table, and on the core router at the Datacenter your box is hosted in.

        http://10.70.90.1/webfig/#IP:DHCP_Server.Leases.new

        Create DNS entries for whatever is going to live there: https://github.com/nycmeshnet/nycmesh-dns/pull/155

        Configure the cluster in Terraform (see this PR as an example) and set up a GitHub action to deploy it.

        Environment Variables

        Accessing the CLI

        To access the CLI, you can copy the kubeconfig off the server if your SSH key is configured to be sent to the nodes:

         scp debian@10.70.90.168:.kube/config ~/.kube/config-gamma1

        Ensure that you change the URL of the kubeconfig when you copy it.

        alias gamma1="KUBECONFIG=~/.kube/config-gamma1 kubectl"

        Deploying Services

        If nothing is happening after you configure your service, it might be because the workflow is invalid:

        https://github.com/nycmeshnet/meshdb/pull/774

        https://github.com/nycmeshnet/meshdb/actions/runs/12173846919

        Back to top