Dev Environment GitOps based Install
This guide is not meant to be a definitive guide to GitOps and
how it can be used with UnderStack or even a best practices example
but instead focused on an example development oriented installation.
It will make a few assumptions and some opinionated choices that may
not align with a production best practices installation.
Most notable assumptions are:
- GitOps tooling runs on the same cluster as the deploy
- AIO (All-in-One) configuration
- Your cluster is a blank slate and can be entirely consumed
You will have the source to your deployment and all the pre-deployment
work will occur on your local machine and not on any of the target
machines.
Getting the source
You must fetch the source to this repo and since we will be using
GitOps, you must also have a deployment repo. These
operations can all happen locally on your development machine.
git clone https://github.com/rackerlabs/understack
# then either
git init uc-deploy
# or
git clone https://path/to/my/uc-deploy
Pre-deployment
Embracing GitOps and declarative configuration, we will define three
distinct pieces of information for your deployment.
- Infrastructure: Where the software will live (TODO: this defines the cluster)
- Secrets: What are all the credentials, passwords, etc needed by the software
- Cluster: The actual software that will be deployed
To properly scope this you'll need an environment name. For the
purposes of this document we'll call it my-k3s.
Environment Variables
To avoid defining many environment variables we'll simplify by creating an
.env file for our deployment. In this case we'll call it my-k3s.env and
place it where we've cloned understack. A complete file would like like
if [ -n "$BASH_SOURCE" ]; then # bash
UC_DEPLOY="$(cd "$(dirname "${BASH_SOURCE[0]}")" && git rev-parse --show-toplevel)"
elif [ -n "$ZSH_VERSION" ]; then # zsh
UC_DEPLOY="$(cd "$(dirname "${(%):-%x}")" && git rev-parse --show-toplevel)"
fi
DEPLOY_NAME="my-k3s"
UC_DEPLOY_GIT_URL=git@github.com:myorg/uc-deploy.git
UC_DEPLOY_SSH_FILE="$HOME/devel/uc-deploy-key"
DNS_ZONE=home.lab
UC_DEPLOY_EMAIL="my@email"
UC_AIO=yes
The final one, UC_AIO is special for the Quick Start guide because it
informs the scripts that you are building an All-In-One system. This is
not a production configuration and is merely there to aid in on boarding.
Paths
The UC_DEPLOY variable is a local path on your machine to where
your deploy repo has been cloned.
Deployment Name
The DEPLOY_NAME variable contains the name that you'll refer to your
deployment as.
Git access for ArgoCD
ArgoCD will need to know where it can access your deployment config
repo. This can be over SSH with a key or over HTTPS or via a GitHub App.
At this time the scripts only support SSH. It is recommended to
use a GitHub Deploy Key, the private key of which
should available locally and the path to it should be set into the
UC_DEPLOY_SSH_FILE variable. While the SSH clone URL for your repo
should be set to UC_DEPLOY_GIT_URL.
DNS for Ingress and SSL certificates
All services will utilize unique DNS names. The facilitate this, UnderStack
will take a domain and add sub-domains for them. The script will also create
a cluster issuer for Cert Manager which will use
the http01 solver by default, so you'll need to provide your email address
which needs to be set into the UC_DEPLOY_EMAIL variable.
All Ingress DNS names will be created as subdomains of the value you put
into the DNS_ZONE variable.
Saving the Environment Config
Getting Ready to Generate Secrets and Configs
You can run source /path/to/uc-deploy/my-k3s.env to have $UC_DEPLOY in
your shell.
Populating the infrastructure
For details on how to configure ArgoCD, see the
Management Cluster page.
Bootstrapping ArgoCD and Sealed-Secrets
If you do not have ArgoCD deployed then you can use the following:
kubectl kustomize --enable-helm \
https://github.com/rackerlabs/understack/bootstrap/sealed-secrets/ \
| kubectl apply -f -
cd path_to_understack_repo
./bootstrap/argocd.sh
Generating secrets
Secrets in their very nature are sensitive pieces of data. The ultimate
storage and injection of these in a production environment needs to be
carefully considered. For the purposes of this document, Sealed Secrets
has been chosen; other tools like Vault, SOPS, etc should be considered
for production deployments.
./scripts/gitops-secrets-gen.sh ./my-k3s.env
git -C "${UC_DEPLOY}" add secrets/my-k3s
git -C "${UC_DEPLOY}" commit -m "my-k3s: secrets generation"
Defining the app deployment
In this section we will use the App of Apps pattern to define
the deployment of all the components of UnderStack.
./scripts/gitops-deploy.sh ${UC_DEPLOY}/my-k3s.env
git -C "${UC_DEPLOY}" add helm-configs/my-k3s
git -C "${UC_DEPLOY}" commit -m "my-k3s: initial cluster config"
Final modifications of your deployment
This is point you can make changes to the ArgoCD configs before
you do the deployment in your $UC_DEPLOY repo. You'll want to consider
any changes to each of components to your cluster by modifying or adding
values files or kustomize patches. This should be considered a rough template
that is yours to modify. Once you've made all the changes you want to make,
ensure that you git push your $UC_DEPLOY repo so that ArgoCD can access it.
For authentication, please review the authentication documentation.
For OpenStack Helm components, an empty file in $UC_DEPLOY/my-k3s/helm-configs
has been created for each component for you to use for customization.
See the Preparing Your Deployment section for component specific details.
Doing the Deployment
At this point we will use our configs to make the actual deployment.
Make sure everything you've committed to your deployment repo is pushed
to your git server so that ArgoCD can access it.
Configure your ArgoCD to be aware of your cluster:
Now configure your ArgoCD to have the credential access to your deploy repo:
Label the node(s) to allow OpenStack control plane installation:
kubectl label node $(kubectl get nodes -o 'jsonpath={.items[*].metadata.name}') openstack-control-plane=enabled
Finally run the following to have ArgoCD deploy the system. We will create our
App of Apps configuration here:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: understack # name for your app-of-apps app
spec:
destination:
namespace: argocd # where ArgoCD will read sources from
server: https://kubernetes.default.svc # local cluster where ArgoCD is running
project: default
source:
repoURL: https://github.com/rackerlabs/understack.git
targetRevision: HEAD
path: apps
kustomize:
# enables updating ArgoCD as well, remove it if you'd like
components:
- appsets/argocd
# optional patch to apply deployment overrides
patches:
- target:
group: argoproj.io
version: v1alpha1
kind: ApplicationSet
patch: |
- op: add
path: /spec/generators/0/matrix/generators/0/merge/generators/1
value:
git:
repoURL: URL_TO_YOUR_DEPLOY_REPO
revision: HEAD
files:
- path: "*/deploy.yaml"
values:
understack_url: '{{ get . "understack_url" }}'
understack_ref: '{{ get . "understack_ref" }}'
deploy_url: '{{ get . "deploy_url" }}'
deploy_ref: '{{ get . "deploy_ref" }}'
syncPolicy:
automated:
prune: true
selfHeal: true
At this point ArgoCD will work to deploy UnderStack.
Advanced Configurations
Argo CD
The UnderStack deploy makes a hardcoded Argo CD config that works with github.com and a SSH key.
If you use something, such as a privately hosted Git service, you can refer to the Argo CD
documentation for configuration details: https://argo-cd.readthedocs.io/en/stable/user-guide/private-repositories/#unknown-ssh-hosts