Vendor Competency Certifications White Paper

The "No Network is 100% Secure" series
- Competency Certifications -
A White Paper

Experience versus having a particular "certification" .. a primer for non-technical Recruiters, Managers and HR types: Consider that you have to undergo a life threatening, vitally important operation. You have a choice between an experienced surgeon who has done this procedure hundreds of times before and someone with little or no experience but who has passed a 250 question, multiple choice, "surgical certification" test. Which would you choose?

Competency certifications: Competency certification is ostensibly an independent verification that the set of competencies or work skills required to perform a particular job are in fact present.

Without certifications, any individual can claim to be an "expert" at anything they like. Competency certification is a process by which you can [theoretically] verify that all the required skills necessary to do a particular job are present. In other words, you have a better tool to determine if an individual is "work ready".

Most jobs have standards regarding the set of competencies necessary to perform all the tasks expected from a person in a particular category. There are three types of competencies:

Critical - essential to the task. If this is not present, the individual can not do the job.
Important - non-critical.
Nice - these are nice to have but not important to the task.

A competency may have a long or a short "shelf-life" dependant upon its nature. Competency in a complex technical process may have a short shelf-life, after which a refresher is required.

Vendor certification: Whether it's Novell, Cisco Systems, or Microsoft, a vendor controlling the certification process on its own products is like a fox guarding the hen house. Certifications do have some value but more so if independent, nonprofit organizations conduct the programs. Vendors are profit-making companies exercising their right to expand branding or franchises to make a few more bucks. The real problem is with recruiting agencies, personnel departments and human resources staff members who have latched on to certifications as an easy, no-brains way to cull their stacks of resumes. As a result, certification has become a required de facto guild license. They provide a false measure of competence that can deny a capable but uncertified person a job. Whereas many with certifications are competent or even outstanding, I've met and worked with many people who are experts but lack the Vendor's "seal of approval" certification. Many firms mindlessly choose a freshly certified boot camper over someone with experience but no certification solely because of the Vendor imprimatur.

Microsoft MCSE: No where is the false sense of competency more obvious than with holders of the Microsoft Certified Systems Engineer (MCSE) certification. The fast track to an MCSE is to take the one week training course and then take the certification exam. Never mind that 95% of Microsoft Windows problems can be resolved by either a reboot or a software reload. Just memorize the study guide and keep taking the test until you pass it. Once you're "certified", add the title of "Systems Engineer" (or Senior Systems Engineer if you are really bold) to your resume and off you go.

The thoughtful reader can probably understand why real Engineers snicker at 20-something Technicians with no job experience calling themselves "Engineers". And "Systems Engineers" to boot!

Cisco CCNA: Another highly over-rated certification (in my opinion) is the Cisco Certified Network Administrator (CCNA). I've long since lost count of the number of CCNA certified "Network Engineers" I've interviewed who had no idea what the differences between TCP and UDP are. Or who could not tell me the difference between a switch and a router. Or who had absolutely no idea how a packet get's on the LAN and travels down the wire. This is really basic stuff for someone who is "certified" to not know. Considering that the passing grade for the CCNA exam is 85%, you have to wonder what questions are being asked if success can be had by people who have no idea what the difference is between a collision domain and a broadcast domain.

Certification Doesn't Guarantee Competency: The big problem with certification testing (in my opinion) is that it is more an exercise in knowing how to answer trick questions than it is about finding out if the candidate knows how things actually work.

For example, almost all certification study guides will advise readers that many of the exam questions will present more than one correct response choice. But the one and only "right" answer (the response that will add to your exam score) is the choice that is the MOST correct. What kind of BS is that?!?! And even more bizarre... some questions will offer no correct choices. In these cases, the "right" answer is the choice that is the least wrong. And if all of that was not convoluted enough, the "right" answer is the right answer according to the Vendor. Reminds me of boot camp where there was a right way, a wrong way and the Drill Instructor's way.

The above lunacy is the reason why people can memorize a 1,000 page CCNA study guide and still have no real understanding of how subnetting works.

Certification exams generally focus on trivia which, furthermore, is typically ephemeral. What is the significance of being able to answer the question, "What hot key in Visual Studio 6 brings up the Replace dialog", when I can simply click on the Edit menu and observe that Ctrl-H can be used. If Administrators should memorize such things, what's the purpose for the Help files and the myriad of Vendor publications? Not to mention Googling for an answer, which is typically what I do if I don't happen to know a particular obscure Cisco IOS command off the top of my head. Mea Culpa, but my 62 year old brain can no longer store the entire man pages for RHEL5.....

In my opinion, the measure of a good Engineer is knowing how to find information, not in having instant recall for every manner of technical trivia that no one...and I mean NO ONE cares about.

CISSP certification: The Certified Information Systems Security Professional (CISSP) certification is one that is gaining popularity and is not trivial to obtain as the ones described above are. I read the 700+ page study guide and I have to say that the scope of the material is quite broad if not deep. The exam is a 6 hour, 250 question affair and even after you pass, you can not be certified until you have shown documented evidence that you have worked with computer security in the "real world" for five years.

However, my complaint with the CISSP track is similar to those voiced above: the exam material focuses on minutia that no "normal" person is going to remember a few days after the test has been taken. An example would be: "Is biometrics a type 1, type 2, type 3 or type 4 method of authentication? IMHO, who cares?

Certainly, anyone passing the CISSP track would have to know a lot of stuff about a lot of stuff. But reading over the study material, I can see how someone could pass the exam and not know anything about syslog or what to look for in investigating security breaches.

I am not completely down on the value of industry certifications and I may actually go after the CISSP at some point. That endeavor would certainly require a substantial investment in time though and time is something that has been at a premium lately. I do have the Network General Certified Network Expert (CNX) certification which I thought was quite the certification to have back when I earned it. I also have a few "lightweight" certifications from Banyan, Novell, Microsoft, etc. since I thought that having them would probably help my consulting practice. But since most ECNEs don't have sense enough to pour water out of a boot (IMO) the only time I talk about my certifications is when I am with a client trying to convince him that I have the skills needed to work on the project he has in mind. There were lots of Novell guys without certifications who knew waaaay more about Netware than I did for me to feel like my ECNE was actually meaningful.

So what does it all mean?: I had an interesting experience recently that is worth sharing here. I was interviewing for an OVO Administrator project at a large public utility here in town. I figured I was a slam dunk for the job since I had already built a NOC for the other public utility here in town. In addition, I am also very proficient with network security including Aurora and SCADA vulnerabilities which, of course, are vitally important to the power generation industry. Being a local candidate was very important to the hiring manager and since I am the only Openview guy in Portland with anywhere near 20+ years of NOC design and deployment experience, I was pretty sure all I had to do was convince them that I was easy to work with and I had the job. But... not so fast Kemosabe, as it turned out.

So I'm asked a "test question" which actually felt a little disrespectful considering that this guy was probably in diapers the first time I started working with HP Openview. But... due diligence, right? I'm not offended and I set out to respond to his question as best I can. So to set the stage: I'm asked a troubleshooting question about what was apparently a serious and difficult to solve real life issue that they had recently experienced. So most importantly, this was not a "theoretical" exercise. It was a problem that they had already shed a few hours of blood over and already knew what the "correct" answer was. And what became blatantly obvious after the fact was that the immediate response they wanted to hear after asking their question was, "Oh... Oracle is broken".

I had maybe five minutes to cough up the right answer and faster would have been better. Additionally, the fellow asking the question had just logged off of Openview and I haven't worked with it for over a year so mea culpa.... I'm just a tad rusty to be trying to troubleshoot difficult problems in a vacuum with a stopwatch ticking away.

Another key tidbit is that I am an expert at enterprise monitoring, not so much an expert at any particular Vendor monitoring product. In NOCs that I build, a key feature is that I aggressively monitor all of the monitoring software so that the NOC doesn't miss any outages because the monitoring software itself dropped dead. I mention this because the failure he was asking me about never would have gone undetected in any NOC that I have ever built. So basically, he's asking me to troubleshoot a problem that would have never happened had the person who designed their NOC known anything about monitoring best practices.

So in trying to troubleshoot this problem in the abstract, I ask a series of "Helpdesk" type questions. Does this or that work? What happens if you do this or that? What do the logs say? At the end of this exercise, it's obvious that the circumstances surrounding this outage cannot exist. There is just no way that this problem could exist given the results of the questions I was asking. So now I have to waste more time (tick, tick, tick) to validate, clarify and amplify the information I had already been given, at least one nugget of which was bogus. And amazingly, these guys are getting aggravated with me!!

So the answer to the test is that the Oracle database died... a fairly obvious outage that having an agent and a SPI on the Oracle server would have instantly detected. Interestingly, while troubleshooting this issue, I asked several questions about SNMP traps and was told that none of that was working either. Needless to say, this lead me away from Oracle since NNM does not use it. I finally asked if anything was going into the trap logs and the response was yes. That led me to the templates and to the database. So.... sitting in a room with no resources other than asking questions, I figured out what was wrong in just a few minutes... but... not fast enough to suit these guys, apparently, since I didn't get the job.

I don't bring this up to whine about it. When it comes to interviews, some days you're hot and some days you're not. Nobody's fault, really. I blew the interview and that's that. The real bottom line issue though, is that during the interview, the hiring manager went on at some length talking about previous Openview "Consultants" who broke more than they fixed, were impossibly arrogant to deal with and/or who seemed more interested in racking up billable hours than in doing anything even remotely related to developing proactive monitoring capabilities. I didn't get a sense from the things I was being asked that these guys had figured out how to tell the difference between the idiots and the people who actually knew what they were doing. Of course my ego might have gotten tied up in all of this too. Imagine that you hire a Plumber to fix a broken pipe and before you allow him to start work he's required to solder some copper pipe for you just to prove that he can. You can probably guess what his reaction to that is going to be.

The notion that I could work with HP Openview for 20+ years and not be aware that Oracle is part of the picture is preposterous on it's face. So one has to ask, how come this manager hired these clucks to work on his Openview environment but felt that I would not be able to handle the job? Just a guess, but I suspect it had something to do with having Vendor certifications and a puffed up resume since by the manager's own admissions, these guys ultimately were not capable of doing even a mediocre job with the Openview product.

An aside is that this job paid $57/hr tops, which is below market for a reasonable UNIX Administrator, much less someone experienced with HP Openview. This project came through an agency and I'm guessing that the client company is probably paying around $125/hr to fill this position. HP Professional Services charges $250/hr for their Openview people and even crappy Openview Consultants charge upwards of $150/hr. So one would think that the agency would jump in and become part of the sales process since I am one of only a handful of experienced independent Openview Consultants here in the U.S.A. and that hiring me for $57/hr would be an absolute steal. But no... didn't happen. On to the next Green Card Candidate with an OVO certification and zero actual NOC designs and deployment work experience.

I do have a "salvage rate" if this utility ever contacts me to clean up the mess that whoever they wind up hiring causes. And I can assure you that it will be "slightly" higher than $57/hr.....

In case you still aren't convinced: I came upon this really difficult [sic] question that was posed on a technical forum recently. According to the OP's profile, he is a Senior Consultant (not a Consultant... a *SENIOR* Consultant), and an Expert in Microsoft Technologies with over 10 years in the IT Industry. Served as a Solution Architect, college degrees and Vendor certifications up the ying yang and blah, blah, blah....

Here's the question and what a toughie it is too....: I need to upload the data files (text) from the remote PC to the web server automatically at scheduled intervals. Web server connection should be based on a valid authentication.

A: Ever hear of SFTP or SCP? WPUT?

Here's another... this time posted by someone claiming to be a "Systems Engineer" having a RedHat certification: I have given Chmod -R 777 /, it means i have given world writeable permission to slash partition and now i m not able to login and the system hangs up on login screen. Any suggestions we be very appreciable.

Yet another, posted by a Cisco CCNA: 192.168.0.255 is broadcasting on my local network but we do not assigned such ip to any PC. Please help what going wrong and how to resolve this broadcasting issue?

A: And these HR guys still require vendor certifications...

About the Author

Frank Saxton is a computer network security engineer and Easyrider LAN Pro principle. Home-based in Portland, Oregon, Frank has been designing remote diagnostic and network enterprise monitoring centers since the late 1970s. Prior to becoming a professional systems engineering consultant in 1990, Frank had a 20 year career in computer systems field engineering and field engineering management. Frank has a BSEE from Northeastern University and holds several certifications including Network General's Certified Network Expert (CNX). As a NOC design engineer and architect, Frank works regularly with enterprise-class monitoring tools such as HP Openview Operations, BMC Patrol and others. In his enterprise security audit work, Frank uses sniffers and other professional grade monitoring tools on a daily basis.

Next in the security white paper series:

How Cyber Criminals will mature over the next ten years
Are you vulnerable to drive-by exploits?
IT employment challenges of the 21st century
Employment reference checking white paper
ISO/IEC 27005:2008 Standard for Security Risk Management
High value sites recent hacks
Firewall White Paper
Password White Paper
Digital Identification Certificates White Paper
Virus White Paper
Ghostnet White Paper
Cryptography White Paper
OpenID White Paper
Intrusion Detection Systems IDS White Paper
Rootkit White Paper
Unnecessary Windows XP Services White Paper
Scareware White Paper
Exaflood Internet Brownout White Paper
Cloud Computing White Paper
Proxy Server White Paper
Personal Computer PC Security White Paper
Phishing White Paper
DNS Poisoning White Paper
Conficker White Paper
SPAM White Paper
Best Practices White Paper
Denial of Service DoS White Paper
Trojan Virus Attacks White Paper
Port Scanning White Paper
Monitoring Basics 101 White Paper
Monitoring Basics 102 White Paper
Monitoring Basics 103 White Paper
Virtual Machine Security White Paper
Aurora vulnerability White Paper
Shelfware White Paper
Outsourcing White Paper

Easyrider LAN Pro Consulting services:

Network Security Audit and PC Tune-up service

- Proxy server installation and configuration
- Enterprise security consultations
- Disaster recovery planning
- Disaster recovery services
- Capacity, migration and upgrade planning
- Build and deploy central syslog server
- Build trouble ticket systems
- Design and build monitoring environments
- Design and build Network Operations Centers (NOC)
- HP Openview, BMC Patrol consulting