Collecting Logs and Debug Information

Istio has many moving parts. When a problem is reported by a customer, it is useful to know the full state of system.

Option A: Run bin/dump_kubernetes.sh

curl https://raw.githubusercontent.com/istio/istio/c80033/bin/dump_kubernetes.sh | sh -s -- -z

Or run the script locally from the root of this repository:

tools/dump_kubernetes.sh -z

Then send the produced "istio-dump.tar.gz" along with your reported problem.

Option B: Manual collection

If you are unable to follow option A and run the script, manually gather the following info:

• State of all pods running in the system. (kubectl get pods -o yaml --all-namespaces)
• services
• Deployments
• Endpoints

kubectl get pods,services,deployments,endpoints --all-namespaces -o yaml > k8s_resources.yaml

• Names of secrets in istio-system: kubectl --namespace istio-system get secrets

• Config maps in istio-system: kubectl --namespace istio-system get cm -o yaml

• Logs from all istio-components and istio-sidecars. This should include both current and previous logs.

  • foreach kubectl logs POD_name -c istio-proxy

• Mixer Logs:

  • kubectl logs -n istio-system -listio=mixer -c mixer
  • kubectl logs -n istio-system -listio=policy -c mixer
  • kubectl logs -n istio-system -listio=telemetry -c mixer

• Pilot Logs:

  • kubectl logs -n istio-system -listio=pilot -c discovery
  • kubectl logs -n istio-system -listio=pilot -c istio-proxy

• All Istio configuration artifacts.

  • kubectl get $(kubectl get customresourcedefinitions -o jsonpath="{.items[*].metadata.name}" | tr " " ",") --all-namespaces -o yaml

Turn on debugging on Istio components

Turn on debug on Mixer

Debug can be turned on my using Ctrl-z on Mixer on port 9876

• Port forward to Ctrl-z kubectl --namespace istio-system port-forward istio-policy-66f9d57f54-kvrl9 9876 Open browser to http://localhost:9876/scopez/

TODO Turning on debug on remaining Istio components