1. Quick Reference — What to Do in the Moment
Find the situation you are in, follow the steps in order, and run the corresponding script on the correct server. Full explanations for each procedure are in the sections that follow.
All scripts are at E:\SQL\SQLScripts on both servers. Run via SSMS or trigger the corresponding Agent job. Each script prints its progress to the messages window — read it. |
Step | Server | Action (Run Agent Jobs) | What Happens |
FAILOVER (primary is down - promote HLA-CONTINGENCY) | |||
1 | HLA-CONTINGENCY | Run [LS] 01 - Failover | Drains queued logs, recovers databases, removes LS config. Databases come ONLINE. This can be initiated by Techs on HLA-CONTINGENCY using the SQLBackupandFTP application. |
2 | Both | Update connection strings/ODBCs | Point applications to HLA-CONTINGENCY. |
REVERSE LOG SHIPPING (restore DR after failover) – After PCLINIMDB984 is back and stable | |||
3 | HLA-CONTINGENCY | Run [LS] 02A - Setup as New Primary | Takes seed + bridge backup. Starts backup jobs. HLA-CONTINGENCY is now primary. |
4 | PCLINIMDB984 | Run [LS] 02B - Setup as New Secondary | Restores seed, applies bridge, starts copy/restore jobs. PCLINIMDB984 is standby. |
5 | PCLINIMDB984 | Verify | min_since_restore should be 0-5 min within a few minutes. |
FAILBACK (return production to PCLINIMDB984 - planned window only ~5-10 minutes) | |||
6 | HLA-CONTINGENCY | Run [LS] 03A - Failback Step Down | Disables backup jobs. Sets databases SINGLE_USER to freeze activity. |
7 | PCLINIMDB984 | Run [LS] 03B - Failback Recover immediately after 03A completes | Drains remaining files, restores WITH RECOVERY, sets MULTI_USER. |
8 | Both | Update connection strings/ODBCs | Point applications back to PCLINIMDB984. |
RESTORE DR (re-establish original log shipping direction) | |||
9 | PCLINIMDB984 | Run [LS] 04A - Restore DR Primary | Takes seed + bridge backup. PCLINIMDB984 is primary again. |
10 | HLA-CONTINGENCY | Run [LS] 04B - Restore DR Secondary | Restores seed, applies bridge, starts copy/restore. System back to normal. |
11 | HLA-CONTINGENCY | Verify | Both databases should show (Restoring...) in SSMS. |
Normal 5-minute latency is expected and healthy. Only investigate if min_since_restore exceeds 15 minutes consistently. |
Full reference document is attached: