Infrastructure: OpenShift (Beta)
On an OpenShift cluster the Kubernetes data source also shows what only
OpenShift has: ClusterVersion and updates, ClusterOperators, routes and their certificates, projects,
DeploymentConfigs and builds, MachineConfigPools and the firing alerts of cluster monitoring. The
Tower reads the OpenShift API every 60 seconds with a service account token
that only has get, list and watch; nothing on the cluster changes.
This view is Beta: it was tested against recorded OpenShift API responses, not yet against a live cluster.
Page: All integrations In Spitfire: /admin/integrations#cloud → Dashboard on your Kubernetes source's row → Cluster view.
Quick install
- Run the Kubernetes command with
ocorkubectllogged in to the cluster, answer the questions.
curl -fsSL "https://<spitfire>/api/v1/setup/ds/kubernetes.sh?t=<token>" -o /tmp/sf-kubernetes.sh && bash /tmp/sf-kubernetes.sh- The script asks This cluster is OpenShift. Turn on the OpenShift view too?; answer
Y(or Enter). It applies the access below, reads the token and the API's CA from the cluster and saves them encrypted. - When you see
OK: ..., you are done. The OpenShift part appears in the cluster view within ~1 min.
Manual install
Run steps 1–5 on a machine where oc is logged in to the cluster as a cluster admin.
- Check that the cluster is OpenShift.
oc api-resources --api-group=config.openshift.io -o name | grep -q clusterversions && echo 'OK: OpenShift'- Create the read-only access: the
spitfirenamespace, the ClusterRolespitfire-openshift, the service account and its token Secret, the binding for cluster monitoring's alerts, and thenonroot-v2SCC for Spitfire's Collector and Tower (they run as user 10001, which the defaultrestricted-v2SCC refuses).
oc create namespace spitfire --dry-run=client -o yaml | oc apply -f - && oc apply -f - <<'EOF'
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata: {name: spitfire-openshift}
rules:
- apiGroups: [config.openshift.io]
resources: [clusterversions, clusteroperators]
verbs: [get, list, watch]
- apiGroups: [route.openshift.io]
resources: [routes]
verbs: [get, list, watch]
- apiGroups: [project.openshift.io]
resources: [projects]
verbs: [get, list, watch]
- apiGroups: [""]
resources: [namespaces]
verbs: [get, list, watch]
- apiGroups: [apps.openshift.io]
resources: [deploymentconfigs]
verbs: [get, list, watch]
- apiGroups: [build.openshift.io]
resources: [builds]
verbs: [get, list, watch]
- apiGroups: [machineconfiguration.openshift.io]
resources: [machineconfigpools]
verbs: [get, list, watch]
- apiGroups: [discovery.k8s.io]
resources: [endpointslices]
verbs: [get, list, watch]
- apiGroups: [""]
resources: [services]
verbs: [get, list, watch]
- apiGroups: [networking.k8s.io]
resources: [ingresses]
verbs: [get, list, watch]
---
apiVersion: v1
kind: ServiceAccount
metadata: {name: spitfire-openshift, namespace: spitfire}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata: {name: spitfire-openshift}
roleRef: {apiGroup: rbac.authorization.k8s.io, kind: ClusterRole, name: spitfire-openshift}
subjects:
- {kind: ServiceAccount, name: spitfire-openshift, namespace: spitfire}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata: {name: spitfire-openshift-monitoring}
roleRef: {apiGroup: rbac.authorization.k8s.io, kind: ClusterRole, name: cluster-monitoring-view}
subjects:
- {kind: ServiceAccount, name: spitfire-openshift, namespace: spitfire}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata: {name: spitfire-scc-nonroot-v2, namespace: spitfire}
roleRef: {apiGroup: rbac.authorization.k8s.io, kind: ClusterRole, name: "system:openshift:scc:nonroot-v2"}
subjects:
- {kind: ServiceAccount, name: spitfire-collector, namespace: spitfire}
- {kind: ServiceAccount, name: spitfire-tower, namespace: spitfire}
---
apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:
name: spitfire-openshift-token
namespace: spitfire
annotations: {kubernetes.io/service-account.name: spitfire-openshift}
EOF- Check the access.
oc auth can-i list clusteroperators.config.openshift.io --as=system:serviceaccount:spitfire:spitfire-openshift | grep -q yes && oc auth can-i delete routes.route.openshift.io --as=system:serviceaccount:spitfire:spitfire-openshift -A | grep -q no && echo 'OK: read-only access'- Print the token (copy it for step 7).
oc -n spitfire get secret spitfire-openshift-token -o jsonpath='{.data.token}' | base64 -d | tee /dev/stderr | grep -q . && echo && echo 'OK: token'- Print the API address and its CA certificate (copy both for step 7).
oc whoami --show-server && oc -n openshift-config-managed get configmap kube-apiserver-server-ca -o jsonpath='{.data.ca-bundle\.crt}' | tee /dev/stderr | grep -q 'BEGIN CERTIFICATE' && echo 'OK: API address and CA'- On the Tower's machine, check that the Tower reaches the API with the token. Put the token
and the API address from steps 4 and 5 in place of
<token>and<api>.-kis only for this check; Spitfire verifies the certificate with the CA.
curl -fsSk -H 'Authorization: Bearer <token>' '<api>/apis/config.openshift.io/v1/clusterversions/version' | grep -q ClusterVersion && echo 'OK: the Tower reaches the OpenShift API'- Spitfire → All integrations → your Kubernetes source → Edit (or Add a source →
Kubernetes for a new cluster).
- OpenShift API address: the address from step 5, e.g.
https://api.ocp.corp:6443 - OpenShift service account token: the token from step 4
- API CA certificate: the PEM from step 5 (empty: the system's CAs)
- Tower: the Tower that reaches the API (step 6)
- OpenShift API address: the address from step 5, e.g.
- Test connection → Save. The test says
OpenShift API okwhen the token reads every part. Within ~1 min the Cluster view shows the OpenShift part.
Apply the Collector manifest only after step 2: without the nonroot-v2 SCC its pods are refused (unable to validate against any security context constraint). If you turned on Also watch every node as a server, that DaemonSet needs the privileged SCC: oc adm policy add-scc-to-user privileged -z spitfire-collector -n spitfire.
The token in the Secret does not expire; delete the Secret to revoke it. A short-lived token works too
(oc -n spitfire create token spitfire-openshift --duration=8760h), but it must be replaced before it ends.
What is read
| Part | API | Without the right |
|---|---|---|
| ClusterVersion, ClusterOperators | config.openshift.io |
not shown, the test fails |
| Routes and their services' endpoints | route.openshift.io, discovery.k8s.io |
not shown, the test fails |
| Projects | project.openshift.io |
not counted, the test fails |
| DeploymentConfigs, builds | apps.openshift.io, build.openshift.io |
not shown, the test fails |
| MachineConfigPools | machineconfiguration.openshift.io |
not shown, the test fails |
| Alerts | route openshift-monitoring/thanos-querier, /api/v1/alerts |
not shown; optional, the test passes |
When the route has no certificate (the router keeps it), the Tower opens a TLS connection to the route's host on 443 once an hour and reads its end date. Each read is bounded (45 s, at most 500 routes and 2000 builds); an unreachable API is read again at the next minute, nothing is buffered.
Findings
| Finding | When | Severity |
|---|---|---|
| ClusterOperator not available | Available = False | critical |
| ClusterOperator degraded | Degraded = True | critical |
| Cluster update failing | ClusterVersion Failing = True | critical |
| MachineConfigPool degraded | Degraded = True or a degraded machine | critical |
| Route not admitted | the router did not admit it (e.g. HostAlreadyClaimed) |
warning |
| Route certificate ending | under 14 days; under 3 days | warning; critical |
| Build failed | the latest build of a BuildConfig is Failed or Error | warning |
Firing alerts are shown, not turned into findings; route them with event policies.
Common errors
- Test:
auth: the token is wrong or was revoked. Run step 4 again and paste the token. - Test:
permission…spitfire-openshift: the ClusterRole or its binding is missing. Run step 2 again. - Test:
tls: the API's certificate is from an internal CA. Paste the CA from step 5. - The alerts line says "not read": the
spitfire-openshift-monitoringbinding is missing, or the Tower does not reach thethanos-querierroute's host. The rest works without it.
Undoing it
Only turning off the OpenShift view (Kubernetes metrics go on):
- Delete the OpenShift objects. It should print
OK: removed.
oc delete clusterrolebinding spitfire-openshift spitfire-openshift-monitoring --ignore-not-found && oc delete clusterrole spitfire-openshift --ignore-not-found && oc -n spitfire delete secret/spitfire-openshift-token serviceaccount/spitfire-openshift rolebinding/spitfire-scc-nonroot-v2 --ignore-not-found && echo 'OK: removed'- On All integrations
In Spitfire: /admin/integrations#cloud, open the Kubernetes source, clear the three OpenShift fields and save.
Removing everything:
- Run step 1 above.
- Remove the Kubernetes source and the Collector as in Connecting Kubernetes → Undoing it (deleting the source in Spitfire included).
- Delete the file the script left.
rm -f ~/.spitfire-setup/kubernetes.state && echo 'OK: local files removed'