Takeaways from the Customer Advisory Board
Filed under: Best Practice, Industry | Tags: Agiliance, Industry
Leave a Comment 
Pravin Kothari
I just returned home from a week in Washington DC where we held a regional Customer Advisory Board meeting and a CISO Roundtable event.
It was great to spend time with some of our customers from up and down the east coast and even one all the way from Utah. Significant number of our customers from various industries including financials, healthcare, federal, Canada, etc. attended the meeting. We discussed our product roadmap and vision and received excellent validation and feedback. Most of the customers presented their deployments and success stories. It was also a great chance for the customers to hear about each other’s projects and discuss lessons learned. Kudos to our customer support and professional services teams, which earned universal high praise from our customers. Some high level feedback and observations from the Customer Advisory Board:
- Plan big but start small: All the customers had a great deployment strategy. They all bought Agiliance’s product because they had a vision of an enterprise GRC platform, a platform that can address all their GRC needs across the enterprise. However, they all had a very realistic deployment strategy. The key is to start with a very precise project with a very precise business benefit. Demonstrate value to the business and gain support for the tool, then broaden the usage. Many of the customers in attendance are on their second and third phase of deployment. Customers typically start with 1 to 2 of their use cases and then grow from there.
- No one can afford customization any more: The impact of this down economy is still very real. Everybody has less resource than before. Customer expectations have clearly changed. Customers want a product that works out of the box, that is easy to use, is easy to administer, and easy to upgrade. No one can justify large consulting bills year-in and year-out, even the large financial services and federal customers who have historically done a lot of customization to the products they buy. Customers want a product that works and not leave the main task as an exercise for them.
- Automation is no longer a nice-to-have: Compliance is still the number one driver for customer’s adoption of GRC technology. The burden of regulatory and contractual compliance is driving customers to automation. Everyone at the meeting agreed that there is no way their sizable organizations can maintain any resemblance of compliance without the help of technology. Resource and budget is just too limited vs. the responsibility they have to meet. Customers want to offload manual, repetitive and non-value-add work from their staff, improve efficiency and focus more on their business needs.
A big thank you to the customers who attended this Customer Advisory Board meeting and provided valuable feedback to help us continue to grow the product and company. I will summarize the key takeaways from the CISO Roundtable event once I get a chance to catch up from my week-long trip.
Is it Out-of-The-Box or Customized Application? Think Upgrade!
Filed under: Best Practice, Technology | Tags: Architecture, Framework
Comments (3) 
Pravin Kothari
Another way to determine if an application is OOTB or Pre-Assembled is to look into the upgrade history of the application. The biggest issue with Pre-Assembled applications is that they quickly turn into custom applications. As business requirements evolve, the customization usually gets more extensive. Why is this not necessarily a good thing? Customization is almost never supported by the vendors. At best, vendors will provide “best-effort” support for area of the application that has been customized. Ultimately, there is the inevitable upgrade challenge. Attempt to upgrade a customized application more often than not turns into a complete redeployment. Redeployment is obviously time consuming and expensive. As I have said many times now, as long as that is a trade-off that is understood upfront and is a desired trade-off, then there is nothing wrong with selecting a RDP based application. Too often, we see companies unknowingly go down the customization path and be surprised with the effort and cost associated with such a strategy.
A customized application is expensive to the customer, and can be equally expensive to the vendor. Patching, new feature design, platform certification all become exponentially more difficult as the engineering team now has to account for more variations and unknowns in the install base. The fact that customers cannot upgrade easily, usually results in the vendor supporting many different versions of the product. Even very old versions still need to be maintained because it is required by important customers. There are numerous stories in the software industry about vendors struggling to support legacy products because all the engineers that worked on the legacy code had left the company. New engineers support the product by reverse engineering the legacy code. The solution to this problem is usually to ask the customer to buy more consulting services or premium support.
Upgrading a customized application can involve dealing with any of the following, ranked in increasing order of difficulty:
- Manual apply the upgrade and migrate your data because vendor’s upgrade tool is not compatible with the customization.
- Re-apply all customizations after automated or manual upgrade
- Update customization due to changes in data schema
- Update customization due to changes to API
- Update customization due to changes in product behavior
- Update customization for past workarounds when new features are introduced
- Update customization that uses undocumented API
- Difference between new vendor code and custom code to figure out what has changed and merge customization into new vendor code
If you are looking for a OOTB product, then you should ask deeper questions about upgradability during the product selection process. The answers will help you calculate the total cost of ownership over the long run. Ask the vendors and check with customers who have been using the product for at least 2 years and have done at least one major version upgrade. Some questions to ask:
- How many different versions of the product are in use? A true OOTB application should have no more than 2 major releases, each with a couple of minor releases still in production.
- How long does a typical upgrade project last? Is it a few days or weeks, or is it more like 3 to 6 months or even longer?
- What percentage of customers use the latest major release? It should be over 50% six months after a major release.
- What percentage of customers has successfully upgraded without using paid services?
One lame response you will often hear from vendors when justifying why their upgrade statistics are less than stellar is that while their software is easy to upgrade, some customers just do not want to upgrade. Every customer I have ever met, especially for emerging market such as GRC, wanted to upgrade to the latest version in a timely manner. Customers who put off upgrading for a long time do it for a simple reason: the pain of upgrading outweighs the pain of living with old software. It’s as simple as that.
Is it Out-of-The-Box or Pre-Assembled? Try before you buy!
Filed under: Best Practice, Industry | Tags: Architecture, business
Leave a Comment Pravin Kothari
I hope I made it clear in my last blog the difference between true OOTB and Pre-Assembled applications, which often require development to meet your needs. Sometime it is very difficult for a customer to tell the difference during the product selection process. Applications requiring customization can be made to look very slick, like a movie trailer. It’s amazing what a good sales engineer can do – they are not any less skilled than Hollywood artists. 
So how can you determine if a vendor is selling OOTB or Pre-Assembled application? The easiest way is to just go with an authoritative source if available. In the IT GRC space, take a look at reports like Gartner’s MarketScope for IT-GRCM. If you want more details, place an inquiry call with the analysts that wrote the research. In Gartner’s case, you would want to talk to Paul Proctor. Remember that analysts like Gartner will never outright endorse a vendor for a number of reasons, however, they can help you determine what type of technology is best for you, whether OOTB or something else. They can also tell you which vendor is really in which category. As part of their research, they talk to many customers who have actually used the products and can give you examples supporting their recommendations. If you have access to these analysts, you should not hesitate to request for an inquiry call.
The most bullet-proof way to validate this is to get your hands on the product. There are many different ways you can do this as well. Proof-of concept (POC) is a popular way. In a POC the customer gives vendors use cases and ask them to come back and show that the tools can meet the use cases. It is debatable how useful is a POC if a vendor is willing to invest in building customization for the POC and represent it as standard product functionality. Another approach is a Sand Box. This requires a vendor to provide an environment to the customer where they can play with the software without the vendor looking over their shoulder. This can be effective if it is done correctly. There are two wrong ways to do this. If you give a vendor detailed use case and other data so they can provide you a custom sand box, then you are not that better off than just a POC. If you just ask for the generic product and play with it without any training, you will likely get frustrated and not be able to explore some of the more advanced functions. Anything you can use without any training probably doesn’t have enough flexibility to meet your needs. The recommended way of doing this is to go through the vendor’s standard training course and ask for a generic evaluation copy for two weeks. Go through a fruit-fly version of the deployment experience on your own and see what exactly is OOTB. Allow a vendor to hold your hands along the way and even make recommendations on how to do things, but don’t let them do anything for you. Some people call this the “touchless POC”. You may have to pay to attend the training and you will have to have technical resource available to support this, but it’s a pretty small price to pay if you see how much effort some customers put into their RFP and POC processes.
Another good acid test is to look into upgradability of the product. We’ll discuss that next time.
Earl Perkins of Gartner Talks About Justifying Security Projects
Filed under: Best Practice, Industry | Tags: business, compliance, enterprise risk, Gartner, risk, Risk Management
Leave a Comment Pravin Kothari
Early Perkins of Gartner wrote a blog yesterday about the challenges of building business justification for identity management products (you can just broaden the whole discussion to security technologies in general). In his point #3 he pointed out that if security folks want to be invited to the Big-Boy table, they need to link security to justifications that business understands, such business risk, compliance and business process improvement. Make sure you read the response from Ed King, one of our resident identity management geeks here at Agiliance. He built on Earl’s thoughts a bit further and offered some concrete examples of risks that can be measured and provided as business justifications for security technologies.
A Hammer Factory
Filed under: Technology | Tags: Architecture, Development, Humor
Leave a Comment Pravin Kothari
All this discussion about OOTB and PDP reminded me about this post from Joel on Software. It is simply hilarious. It also does a nice job of boiling down my last 2 blog entries to “Are you looking to buy a hammer or a hammer factory?” A little humor before the weekend.
Out-of-The-Box vs. Pre-Assembled
Filed under: Industry, Technology | Tags: Architecture, Framework
Leave a Comment Pravin Kothari
In my last blog entry I made the claim that “a product cannot be both OOTB and RDP”. Some of my readers objected to that claim and used SharePoint as an example of a product that is both OOTB and RDP. This deserves some more explanation.
Yes, it is absolutely correct that a software product can be both OOTB and RDP. SharePoint is a great example. Others examples are Lotus Notes, ERP, identity provisioning, and help desk products. With these products, you have lots of rich features you can use OOTB with great configurability. These products also offer you a RDP framework and tools to extend and build your own implementations.
Note that “configurability” is different from “customizability”. All enterprise-class products, including OOTB products, are required to provide configurability and personalization (all via the user interface) to adapt to customer needs and environment. While the ‘customization’ term indicates deeper changes, expensive services, or even programming to build your own implementation that is typically more difficult to maintain and upgrade.
There are many shades of gray here about how OOTB really is OOTB? On one end of the spectrum there are products that many customers use with just configuration to meet their business needs, never have the need to use the RDP capabilities. SharePoint is such an example. On the other end of the spectrum, you have ERP applications that almost no one uses OOTB – almost all ERP applications in production have heavy customization that takes months and years of services to deploy and then again to upgrade. So what should we call products that have a lot of OOTB capabilities but almost all customers still have to “enhance” them using RDP functions? I would rather use the term “Pre-Assembled” products to make the distinction clear from OOTB products. A Pre-Assembled product is simply a starting point for you to build on or finish off (depending on if you are a glass-half-full or glass-half-empty kind of person ).
An analogy here is like buying a storage shed for your backyard. You can go to your local hardware store to buy a shed in a box. There are inexpensive ones that are pretty basic and there are nice ones with more features and options. People buy the ones that fit their needs and budget. More often than not the shed you buy doesn’t have every feature you would like to have or every feature designed just the way you like it, but it meets 90% of your needs and wants without a lot of hassle and effort on your part. This is buying something that is quite literally OOTB. I have 3 of these OOTB sheds in my backyard. The other way is to go to a local custom shed shop and buy something that is tailored to your needs and wants more precisely. The shed shop has many pre-fabricated parts and assemblies you can choose from to make your final shed. They even have sample designs that you can use as a starting point or get ideas from. Most customers use one of these suggested designs as a starting point for their grand design. The shed that ends up in the backyard is customized for you. This is buying a RDP product that is Pre-Assembled. Now, can you go to a custom shed shop and buy a Pre-Assembled model as-is without any customization? Absolutely! Do most people who buy a shed from a shed shop do that? No.
Fortunately in today’s IT GRC world, there are not that many shades of gray in terms of “OOTBness”. You basically have the true OOTB products and RDP framework based Pre-Assembled products. As I said in the last blog entry, there is no single right technology, as long as you understand the pros and cons of each approach and make an informed decision based on your needs and resources. You should always look for “configurability” for flexibility, but you don’t want to involuntarily end up owning a “customized” one-off implementation.
BMC Viewpoint’s Special Issue on IT GRC
Filed under: Annoucements, Best Practice | Tags: compliance, risk
Leave a Comment Pravin Kothari
BMC’ Viewpoint magazine’s latest issue is all about IT GRC. It has some interesting articles from contributors inside and outside of BMC. I wrote an article titled “The Big Picture: Beyond Compliance to Risk Management”. You can check it out by request a free copy from BMC here: BMC Viewpoint
Broken Promises of Privacy: Responding to the Surprising Failure of Anonymization
Filed under: Industry, Technology | Tags: Personal Identifiable Information, privacy
Leave a Comment Pravin Kothari
I ran across this fascinating paper today on privacy by Paul Ohm of University of Colorado Law School. It does a wonderful job of describing landmark researches on the techniques of “reidentification” and their implications on privacy laws, with real-life case studies on AOL, State of Massachusetts and Netflix. The fundamental assumption of the effectiveness of de-identifying data and removing Personal Identifiable Information is now questionable at best. As data mining technology and social networking database continue to develop, as well as computing power become cheap and endless with the development of cloud, it is not difficult to see that current de-identification techniques can become virtually useless. PII will eventually cover every piece of data that can remotely contribute to a “digital fingerprint”.
With such information and technology at our finger tips for good and evil, it is increasing difficult to imagine our privacy to be ever completely safe. From an organization standpoint, it is also evident that privacy protection will become an increasingly difficult challenge. Privacy event/breach is virtually inevitable, so “effort” becomes the key differentiation between whether that privacy event becomes a disaster or a slight incident. Organizations that cannot demonstrate investment in and due diligence for a privacy protection program is simply gambling with their brand reputation.
What exactly is “Out-of-The-Box”?
Filed under: Industry, Technology | Tags: Architecture, Development, Framework
Leave a Comment Pravin Kothari
In Gartner’s latest MarketScope for IT GRCM in Q2 2009, Gartner did a great job identifying the two very different approaches taken by vendors in this space. Gartner called these two approaches OOTB (Out-of-The-Box) and Rapid Development Platform. Here is exactly how Gartner described them in the document:
“OOTB products are designed to work without significant customization or configuration beyond entering the control, asset and risk content specific to the enterprise. OOTB products are characterized by vendor-provided content for templates, workflow, sample controls and mappings.”
“Rapid development platform products are characterized by flexibility, which means they may be more-closely aligned with customer requirements, but they may also require services to customize and configure…. Organizations should beware of products that can do ‘anything’ except for very little OOTB without consulting assistance.”
The leading vendors in this space are evenly divided in the two camps. In case you are wondering, Agiliance RiskVision is in the OOTB group.
Like so many things in enterprise software, all is well until the marketing folks started to abuse the words (I can already hear my marketing VP coming down the hallway… he reads my blog ). If you go through the various vendor’s web sites and product literature, practically every vendor is claiming their products are OOTB, because it sound so good to the customers. In fact, many of the RDP (Rapid Development Platform) vendors have gone out of their way during the sales cycle to make their product demo like OOTB, while still touting the infinite flexibility of their product over the “other” OOTB vendors. Oh yes, they’re trying to have their cake and eat it too.
There is not a single right approach. You can’t say OOTB is better than RDP or vice versa. It’s all a matter of fit. Each customer must first determine what type of organization are you? In order to determine which approach is the right fit for your organization, ask yourself these questions:
- How much technical resource do you have to build and maintain the solution?
- What is your budget for both initial deployment and on-going maintenance?
- How much time do you have for delivering the results?
- How often do you expect to upgrade and patch the system?
- How extensive is your deployment? How many integration points do you have?
- How much flexibility do you need from the product?
- Do you have any flexibility to adopt best practices or do you have to replicate your existing manual processes (or legacy processes) exactly?
While there is a no universal “right” approach for all customers, there is a universal truth: A PRODUCT CANNOT BE BOTH OOTB AND RDP”. These are mutually exclusive approaches to product design. In the next few blog entries, I will drill down a little bit into this topic and would love to hear your feedback on this topic.
Risk Mitigation and Remediation Automation – Let’s Not Reinvent The Wheel
Filed under: Best Practice, Technology | Tags: Assessment, Automation, mitigation, remediation
Leave a Comment Pravin Kothari
We’re finally at the end of this discussion on defining GRC automation. We discussed mitigation as part of the process automation earlier.
Risk mitigation and remediation are not only an integral part, but also usually the most expensive part of a GRC program as it requires changes in your environment. Some of these changes may require new processes and even additional people if the process is not automated. Automation of GRC processes is still relatively new to most companies. Most companies are still working on front-end process automation. More mature organizations have started working on back-end automation for mitigation and remediation of issues. The cost savings from automating the back-end processes can be very significant.
When the risk, gaps and issues have been identified and prioritized, a response is needed to mitigate the risk , this can include requesting an exception, accepting the risk, transferring the risk, providing compensating controls, launching a campaign, etc. Some of these mitigation actions, e.g. requesting an exception, can trigger their own automated processes. Remediation is one of the possible mitigation actions.
Remediation actions can be manual, partially automated or fully automated. Partially automated remediation actions have some parts accomplished with integration and other parts manually executed. For example, automating ticket integration with Remedy, where appropriate user makes the changes by hand. The technology here is integration to the help desk of your choice. Not a trivial task, but not rocket science either.
Certain remediation can be fully automated and it typically involves integration with some sort of management software. System management software, or the new fancier term “business service automation (BSA)” can apply patches, make configuration or policy changes directly to applications, servers, network and telephony hardware. These applications are the bread and butter of datacenter operations team. Security software such as identity management, secured configuration management, data leakage prevention are able to implement access changes via remote requests as well. Identity management can even reach into ERP applications and implement segregation of duties controls on business transactions. These are the tools of trade for security operations team. There are a number of new and old technologies that fall into these two categories of system management and security tools, and there is no sense for me to inventory these on this blog. What is important to note is that these technologies do not fall under the “GRC” management space. Remember that GRC (capitalized) is a well defined application category, where as “governance, risk and compliance” is used loosely to describe a whole slew of technology solutions that have something to do with compliance and risk…. [The abuse of the term GRC by vendors is a whole topic of its own. I’ll reserve that for venting some other day J] System management and security management software are necessary in their own right, solving operational and compliance problems. The key is for GRC applications to leverage these existing assets and not try to re-invent the wheel by duplicating capabilities already available in the infrastructure.
Whew! That was a long discussion, longer than I expected, actually. I hope you found this thread on GRC automation definition useful.
