921ceb6ce0
All checks were successful
Blog Deployment / Check-Rebuild (push) Successful in 6s
Blog Deployment / Build (push) Has been skipped
Blog Deployment / Deploy-Staging (push) Successful in 11s
Blog Deployment / Test-Staging (push) Successful in 2s
Blog Deployment / Merge (push) Successful in 9s
Blog Deployment / Deploy-Production (push) Successful in 10s
Blog Deployment / Test-Production (push) Successful in 3s
Blog Deployment / Clean (push) Has been skipped
Blog Deployment / Notify (push) Successful in 3s
92 lines
1.6 KiB
Markdown
92 lines
1.6 KiB
Markdown
---
|
||
slug:
|
||
title: Template
|
||
description:
|
||
date:
|
||
draft: true
|
||
tags:
|
||
categories:
|
||
---
|
||
## Intro
|
||
|
||
Why I’m sharing this story
|
||
|
||
A quick disclaimer: not everything went smoothly
|
||
|
||
## The Calm Before the Storm
|
||
|
||
My OPNsense setup before the incident
|
||
|
||
Confidence in my homelab
|
||
|
||
## The Unexpected Reboot
|
||
|
||
How I first noticed something was wrong
|
||
|
||
The kernel panic that started it all
|
||
|
||
## First Troubleshooting Attempts
|
||
|
||
Caddy refusing to start
|
||
|
||
Digging into logs and finding corrupted certs
|
||
|
||
Quick fixes to restore services
|
||
|
||
## Filesystem Corruption
|
||
|
||
Upgrade attempt from 25.1 to 25.7 gone wrong
|
||
|
||
pkg errors and a broken package database
|
||
|
||
Discovering the dirty filesystem with fsck
|
||
|
||
## When Things Got Worse
|
||
|
||
Broken bootstrap and missing core components
|
||
|
||
Realizing the system was half-upgraded and unstable
|
||
|
||
The tough decision: reinstall vs. repair
|
||
|
||
## Starting Over the Hard Way
|
||
|
||
Pulling the box out of the rack
|
||
|
||
Preparing the installer USB
|
||
|
||
Fresh install of OPNsense 25.7
|
||
|
||
## The Lifesaver: config.xml
|
||
|
||
Copying my configuration from backup
|
||
|
||
Restoring the system to its former self
|
||
|
||
Which services came back instantly
|
||
|
||
Which ones didn’t (goodbye UniFi controller backup…)
|
||
|
||
## Lessons Learned the Hard Way
|
||
|
||
What I did wrong (and why it hurt)
|
||
|
||
What I should have done differently
|
||
|
||
The single most important file in OPNsense
|
||
|
||
Why keeping off-box backups matters
|
||
|
||
## Moving Forward
|
||
|
||
My new backup strategy
|
||
|
||
Plans to improve reliability in my homelab
|
||
|
||
Final thoughts: sometimes starting fresh is the cleanest fix
|
||
|
||
## Conclusion
|
||
|
||
How this failure taught me more than a normal upgrade ever could
|
||
|
||
Encouragement for others to prepare before disaster strikes |