What to do if Looker dashboards become inaccessible due to a filled temporary storage directory?
Issue Description:
Users might encounter an issue where Looker dashboards become inaccessible. This can happen due to the temporary storage directory (/app/worker/.tmp) getting filled up. The frontend error message encountered is "dashboards not accessible".
Root Cause: A bug within Looker causes the temporary storage directory, specifically located at /app/worker/.tmp, to fill-up. This results in an exception in the Looker container, making the dashboards inaccessible.
Resolution:
- Exec into the Insights-insightslooker deployment located in the UiPath namespace using the command:
kubectl exec -it deploy/insights-insightslooker -n uipath -- /bin/bash
- Once inside the pod, first check if the directory is full using the following command:
du -sh /app/workerdir/.tmp
- This will display the size of the directory. If it's near or at capacity, proceed to the next step.
- Remove the /app/workerdir/.tmp folder with the following command:
rm -rf /app/workerdir/.tmp
- Allow approximately 5 minutes for Looker to reinitialize. After this period, the dashboards should become accessible again
- If the pod deployment still appears degraded in Argo, resync the Insights app. This can be done within the Argo UI or by using the appropriate Argo CLI command.