Playing with Solarwinds Orion NPM – How to recover from a corrupted database

I can’t believe it’s been that long, but I recently realized that my Solardwinds SCP has actually slipped. The SCP was one of the first certifications focused on network management and, as I’m sure you can imagine, I was in there as an early adopter. The training was really good ( I still miss Josh Stevens!) and the test was one of the best tests I’ve every taken in IT. It had some REALLY evil questions on there. You know the kind… the ones that prove you either know your stuff or you don’t.  No messing around with ambiguities. Ahh… good times.  On to the present though.

Open Disclosure

I’m assuming because of the major focus of my blog is network management, I was approached by Solarwinds and offered an NFR license for a couple of their products to run in my labs. As with them, I think it’s important for my readers to understand that I work for HP and sometimes find myself in competition directly against these products. I do also find myself giving some guidance to customers who are using Solarwinds products and trying to manage their HP Networking products through the Orion console. It’s the experience of using NPM and NCM to manage HP Networking equipment that I’m going to try to focus on.  Please don’t ask me to compare products.  I told you where I get paid and you can guess what my official opinion is going to be. 🙂

Orion NPM

The first product I wanted to play with is Solarwinds NPM. Solarwinds has a great following and has been around for a lot of years. There were some things that I really didn’t like about this a few years back when I passed the SCP and it will be interesting to see how the product has improved overtime and whether my old issues have been fixed.

Specifically, I was never happy with the half-enabled web-console.  The fact that I had to bounce back and forth between the windows console and the web browser to get anything done was frustrating to say the least. I know there were a lot of improvements made in Orion 10, and I’ve heard good things about 10.5 specifically.  I downloaded 10.5 and will be upgrading to the 10.6 with hot fix 3 tonight. I’m really excited to see the improvements that Solarwinds has made in the years since I last had my hands dirty with the platform.  WIsh me luck!

Before we get started…

So this is detailing some the issues I had getting NPM up and running.  To say the least, I had some issues. ( as detailed below ).  I’ve written down the symptoms and the fixes that I went through, but to be honest, this was just a REALLY bad Windows build. Sometimes, there’s just nothing you can do when the base operating system gets corrupted right from the initial install.

 

Installing

To be honest, I had some issues getting it running. The licensing actually crashed and somehow it was assigned in the Solarwinds system, but never applied to my system. I also made the mistake of downloading the package that didn’t have SQL installed ( wasn’t clear and I didn’t read the documentation closely enough ).  On the bright side, Solarwinds support actually helped me through this one in about 24 hours. Sometimes thing happen during an install, so I can’t complain too much. Plus, I should have paid more attention when flipping through the documentation. My bad.

Unscheduled Interuption.

Ahhh… well… Sometimes things don’t go as planned.  I had an unscheduled power outage tonight and it seems something has gone wrong with my installation.

NewImage

Google didn’t come up with anything. So I’m off to follow the SQL Management studio where the SolarWindsOrion database is marked as suspect…. hmmm… that’s not good.

A couple of scooby snacks and some super-sluething later and I come up with this link

In a nutshell, it looks my SQL database has been corrupted somehow and it’s now showing up as suspect in the Microsoft SQL server management console. ( While I was banging my head against this problem, I didn’t take a good screen capture. So this is where I ask you to imagine a big yellow exclamation mark of DOOM over the SolarwindsOrion database in the following image.  )

NewImage

Looks like the power outage REALLY messed up the SQL database.  But GoogleTechnician to the rescue!

Solarwinds Configuration Wizard – Attempt #1

So now I’m off to the Solarwinds Configuration tool ( on the console of the windows server ). For this attempt, I run the database configuration only. Thinking, I’ve got a database, issue, let’s just run the database configuration wizard and that should fix it, right?

NewImage

Nope… doesn’t look like this is going to work either

NewImage

Solarwinds Configuration Wizard – Attempt #2

So now I’m off to the Solarwinds database.  Hmm.. nothing on this error.

At this point, I just try what any good network guy does. I start clicking things and seeing if anything will work.

So this is what I did

  •  Logged into the Microsoft SQL Management console and reset the password on the SolarWindsOrionDatabaseUser account to something I knew.
  • Re-ran the Solarwinds Configuration Wizard. This time, instead of just the database, I’m going to re-run this for the Database, Web Site, and the Services.

note: Normally at this point, I would pull the plug, call the patient dead and re-install. But this was supposed to be a learning experience, right? We’re certainly learning now, aren’t we?

NewImage

Look like I’m back in business! Good to go right?

NewImage

Nope. now it’s time to remove the license, delete the VM and start from scratch. I don’t want a known corrupted system monitoring my network, even in a lab.

Hopefully, this blog will help someone with a production Solarwinds deployment who gets this same nasty SQL suspect database error.

Lesson to Learn

In a lab, sometimes things happen. Take the opportunity for the full learning experience when things go wrong. It’s always fun to see if we can bring a system back from the dead. But remember, once you’re done with the learning. Scrap it. This is not the system that I want to be evaluating as I will always be wondering “Hmmm… I wonder if this is normal or if this is a result of that bad install.”

Things go wrong. Known good clone images just have something funky. I’ve seen registry issues on brand new windows installation. SQL strangeness etc… None of which I feel like dealing with for longer than necessary. With how easy it is now to deploy a new VM from a template. There’s just no need to subject myself to this kind of long term pain.

So before I go to bed tonight, I’m going to start cloning a new windows image so that I can re-do the entire install tomorrow night on a clean VM.

FOR THE RECORD :  I’m 100% sure this is not a normal Solarwinds Orion NPM installation tale. I just happened to be the lucky one who was hand-selected by the universe as it thought ” Hmmm…  who can I REALLY mess with today? “.

Can’t wait for tomorrow.

@netmanchris

My iPhone Corrupted Profile or There and Back Again

My journey into the hallowed golden jail of Steve Jobs started in about 2008 with the brand new shiny iPhone 3G.  I was Palm guy through and through. I had started with a Palm II in about 1998, I wonder a Palm V, a 7, an m130, and then moved to the Palm Treo 650 and then 680. But the iPhone was new and this great concept called apps. 

 

For the record, Palm had those first, but they just never had the marketing down to make it real. Not to mention, the data service in those days was just to slow for anything useful.  

 

I bought into the 3G, upgraded to the 3GS, then to the 4, then the 4S.  Along the way, I’ve bought an iPad one ( still running! ), an iPad II ( now owned by a thief in Spain ), an iPad III ( owned by a thief in Houston) not to mention an couple of Macbook Pros, Apple TVs, and Apple Mini. 

So a few weeks ago, the unthinkable happened. My profile in my iPhone 4s corrupted.  What do I mean corrupted? I mean no data access if I’m not on wifi.  

Imagine, going for hours of the day without internet access. And even worse, imagine travelling without network access unless you’re at the hotel. My boss actually threatened to come down to Canada and smash my phone since he “couldn’t get ahold of me”.   This also meant that SMS messages weren’t getting through etc…  

 

Sucks to say the least. 

So I finally bit the bullet and decided to start my entire Apple iPhone experience from scratch. I bit the bullet, plugged in my iPhone to my trusty macbook and clicked on that RECOVER button. 

 

You know, the one that you look at and just shiver in horror as you imagine rebuilding everything in your phone from scratch? 

 

Post-Wipe

 

Basic info:

 

So this was a lot less painful than I imagine. The actual processor connecting to my iCloud account was, as expected, seamless. My contacts, calendars, etc.. all came back as expected. 

I did have to recreate my entire favourite list, which was probably a good thing as there was a couple of people who didn’t belong on their anymore. 

 

Applications:  

This was the unexpected benefit here. I had a bunch of applications on my phone that had been there for years. And to be honest, I was keeping them on their “in case I needed them”. They were taking up space and I hadn’t used some of them in years. I wasn’t prepared to delete them, but I’ll tell you when it came time for me to reinstall them, I was a lot more ruthless in my selection process. I probably gained an extra gig or two in space on my phone.

 

Text Messages

This was the one that hurt. I had text messages on my phone dating back to 2008. I had over 9,000 text on my phone and like everyone else I know. There was a ton of data in there that I had “just in case”.  Picture people had sent me that I never downloaded into my photo app. Messages from people who aren’t in my life anymore. Messages from my wife, etc…  I’m basically a data hoarder, so I’ve got everything.  It was just PAINFUL for me to walk away form all that history. Thankfully, google to the rescue and I found a few different apps that claimed to be able to recover data from the iPhone backup and move it over to a CSV file.  

 

Ok.. one step forward, but not what I was looking for exactly. 

 

And then I stumbled upon SMS Backup Transfer which promised to do exactly what I was looking for. The application promised to retrieve all the text messages from my iPhone backup, and then allowed me to restore JUST the SMS database back to the a new iPhone without any of the other corrupted profile information that was giving me all the trouble.

 

The verdict

 

This software delivered on everything it promised.  I was able to easily import my SMS database from my previous iPhone back and just click on the “restore to iPhone” button and all my data is back. Safe and Secure for a low low price of $25. 

Sure, that may seem a little steep for some people, but I’ll tell you it was worth it to me to have all the data back.

The SMS database was the only thing that I actually had any problems with and this was solved with about 10 minutes on google and a small purchase. Now I just hope the phone continues to run for another few weeks until my iPhone 5s arrives.  On the bright side, I’ll be starting life with my 5s with a new born backup that doesn’t have 6+ years of legacy upgrade data in it.