This is part three of a four-part series from Christopher Gillespie. Read his introduction here and his piece on the AV bill of materials here.

Years ago, maintaining and reporting on the inventory was a matter of many spreadsheets, maybe a home-baked database, perhaps it was some hybrid, or maybe it was Crestron Fusion or Extron GlobalViewer or the like.

Nowadays, this is not enough: We live in a world where IT provides services and rules and guidelines are standardized as to expectations. We need to know where everything is, we need to know its value, we need to create incident tickets when things go wrong and we need to perhaps use catalog items to record that we audited/tested the room on a regular basis. But most importantly, the AV “stuff” can no longer sit in its own realm. All these tasks listed above (and many more) and the various dashboards/reports need to live inside the same system as all the other IT functions. And this will be the Information Technology Service Management (ITSM) platform that we are using. Whether its ServiceNow, Remedy, Oracle, Jira Service Desk or some other solution, we must have a framework in place to be able to capture and track the rooms and all the IT-related data around them. Even in cases where there is not an “official” ITSM platform in place, it’s still critical to think about that future state so that when one is deployed, we have a clear vision in place to ensure that we can start off on the right footing. And more importantly, we have the data in place to be able to just port it over. And even if for whatever reason, there is no system going to be going in… we still need to track all this stuff in order to ensure that the third category of stakeholders (management/finance) is able to get their needs met. And honestly, this is about keeping sane on a day to day basis.

While this may be new to some, or old hat to others, it’s not universal. I know of one VERY large company that is still trying to figure all of this out.

The first step in all of this is to start planning as we would in any project. We have to start with the basics and work our way up from there. The eventual plan is not important at this point, but the planning process is. It is the most valuable part. In the military, planning is a bread and butter task. And as such there are a million adages about planning, but one of my favorites is this: There is nothing as useless as a plan, but there is nothing as useFUL as planning.

In short, the plan will change a hundred times, but a good planning exercise will prepare you for those changes.

Let’s look at a rather interesting quote from way back in 2002. Donald Rumsfeld was talking about the ill-fated search for Iraqi weapons of mass destruction at the time, but his quote stands alone as a mantra for large-scale future planning. When we first read it, it almost sounds nonsensical, but upon reflection, it gets more and more clear. Here’s the quote:

“As we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say, we know there are some things we do not know. But there are also unknown unknowns — the ones we don’t know we don’t know.”

There have been books written about this quote and the simple brilliance therein. It is crucial to think like this when planning a long-term vision like your ITSM platform needs.

Let me break it down with a few examples.

• There are known knowns

1. 1. We know without putting too much thought into it that we need to do certain stuff as it’s a part of our wheelhouse.
   2. We need to track rooms and details related to those rooms.
   3. We need to track equipment.
   4. We need to track when we install/repair/replace stuff.
   5. We need to track issues/incidents.
   6. We need to track activities including:
      1. Proactive room audits
      2. Room validations when they come online or get repaired etc.
      3. Events that might be hosted in a given room

• There are known unknowns

1. 1. We know finance will want to know stuff about the room. But until you go through that exercise… you will have no idea the extent of the info that they want. (Trust me on this one!)
   2. We can imagine that at some point we will need to maybe track historical info, such as:
      1. What was initially installed in a given room?
      2. How many times has said room been refreshed? (When going through this exercise, we realized we had about 300 rooms with no info as to when it was initially installed. We got lucky in that an incredibly smart team member — thank you, Cindy Todd! — had saved a PM database from six years prior when the install was done!
      3. We can imagine that at some point in the future, we might get a random call asking for the IP address of some innocuous piece of gear, or a firmware version.
      4. We know that it would not be outside the realm of possibility that someone might ask, “How many sharp TVs do we have/have had worldwide?” The finance people are looking to maybe get a better deal if we globally standardize on one brand.”

• There are also unknown unknowns

1. 1. I had a person ask me for a breakdown of digital signage players and the native language per location. Certainly hadn’t thought of that when planning!
   2. I got a call asking for the total heat load for a certain IDF closet.
   3. ???

All of the items above are easily dealt with IF we captured the right pieces of info when we initially installed the equipment. If not, well, then we are spending a weekend or four digging through AV closets all over the place to capture the info.

The point is that we can’t just be sitting comfortably expecting that we know everything we have to capture. We have to go through an exercise of long-range planning in order to try to set up a framework where we capture everything we can from the start so that we are well set-up for both the known knowns and the known unknowns. If our mindset was right, and we gathered a diverse group to help formulate those answers, then we will be better suited for the future.

In general, when we build out our ITSM landscape, there are four main things we need to focus on. Skip one of them and then in a year or two, you’ll absolutely be starting all over yet again:

• Conference room inventory AUTOMATION 
• Detailed room information (independent of whatever technology we put in the rooms)
• Inventory management (barcodes and depots)
• Various tickets to create against the room (the easy part)

In this planning, the main hurdle will always be how we track the inventory of rooms. Keeping a conference room inventory accurate over time is an almost nightmarish proposition. This is because of the insane level of flexibility we need along with the disparate systems that are involved in that accuracy. To date, I have yet to meet anyone that has a conference room inventory of over about 200 rooms that can provide a precisely accurate accounting of those rooms. Things like EXACT number and various details about those rooms is always in terms like “about 400” or maybe “in the area of 700,” and so on. Why is this? Let’s look at the stages of the problem:

Adding and removing rooms, spaces or locations.

The first part of the issue is defining what we need to track. At one company I worked at, we called the “room” a COMPUTER ROOM in the ITSM system. And then we subclassified it as an office, conference room, event space, etc. At another company they just called it LOCATIONS. And I now feel that this is a better way to go. Because if you think about it… we put this stuff everywhere. Cafeterias, offices, lobby areas, hallways, conference rooms, patios, parks… it’s not just a room. We had a location at a previous company that was defined by the fact that there was a canoe hanging from the ceiling. “Meeting is in the canoe!” (I’ll give you one guess as to what country this was in.) But the point is that we need to stay flexible and just have classifications for what type of space it is for our reporting needs.

Back to adding and removing: there will always be two types of locations that we have to deal with — defined spaces like rooms and offices. These are easy to add in theory. Just like we integrate with LDAP to keep the inventory of people accurate, we can do so with conference rooms and offices. You just set things up to integrate with exchange (or whatever backend you are using) and there you go. All done. Except… the folks in real estate might be using another program even before that where we need to capture data long before it gets to exchange. Programs like Serraview or Campus come to mind. There are many reasons why other groups may use those platforms, but where things get difficult is that not all APIs are created equally. If you are using ServiceNow, it may not be able to talk directly to that application, so the solution for you might be totally different than it is for someone else depending on their software landscape.

In order to build a robust system, you need two-way data flow between all the client systems and the source of truth.  And this is not always possible. So there will always be a big exercise for your ITSM team to look at how all your applications talk and manage data in order to determine which one will be the source of truth and how the data flows from there. But if you do not have an automated system to add and remove locations… you will have serious ongoing issues. Just like a house needs a solid foundation… so does your ITSM planning for AV. As a result, the largest dragon that we have to slay first… is ensuring that there is some sort of automation that feeds/updates conference room inventory from whatever real estate is using as their source of truth into the ITSM platform. IF this is not happening, most continuing efforts will fail over time. As new buildings come online, go offline, etc., if the place where we are planning to track inventory does not have all the rooms, or has rooms that no longer exist, it will not take long for any dashboards/reporting to start being wrong. Even when going through a planning exercise, in most companies, you will have to draw a line in the sand and caveat any resource models with “this is based on October 12, 2018” because in the next month while you are working on the plan, you could have two campus refreshes and two new buildings come online (or far more). If you don’t have that room inventory automation built… you will 100% be starting over in a few years.

This is of course just the issue related to the inventory of rooms. We will also have other issues related to rooms that change: an office becomes a huddle room, a medium conference room becomes a war room for the new product launch over the next four months. Or another fun trend — rooms that are available to teams locally, but not bookable. At one company I worked at, we had a building with about 120 rooms and something like 80 of them were team rooms. This meant that they were nonbookable and not in Exchange as a reservable space. This not only affects that dataflow potentially, but also (back to the BOM for a sec) how we manage to try to do one-touch meeting starts.

I could continue, but it should be clear by now that this is a big issue. And we need to ensure that we get it right to start, moreso because over time we will have even more systems that we want to communicate to automate more things. As one example, anyone who has ever had to take a room offline for maintenance knows how hard this can be. You have to find all the bookings, reach out to the reservation holders, find them alternate rooms, etc. This is a very painful but necessary thing. I can imagine that in the not-too-distant future there will be product offerings that will drive these sorts of things using AI/ML to assist. And we could potentially trigger it by flagging a room in our ITSM as broken. Who knows? All I do know is that we need the data, and we need it to be accurate. There will of course be things to manually add. Lobbies, that dashboard monitor in the hallway and all the other random stuff, but they are the smallest piece of this puzzle.

A company I worked for suddenly had a five-alarm emergency: We had to have a full audit of all our conference rooms completed in an absurdly unrealistic time frame. My director kept insisting that this would be simple to do as they had done one a few years previously over a few weeks.

I kept trying to get it across that yes, I understood, but the reason we needed to do it AGAIN (ignoring the fact that nobody even knew where the spreadsheet was, and the number of rooms was now double) was because that foundation had never been established. And without the room inventory being automated, we could do it and then it would fail again in just a few years. Alas, it is now clear that I had to find a much better way of explaining that.

Detailed room information

Once we have the locations in the ITSM platform and automated, then the fun starts. There is a simple question to ask… What might we want to know about the locations? The answer to this question is different for every different group of stakeholders you might ask, so finding ways to filter and categorize every space is key. At some point, you will get asked to generate a report of all the digital signage installations per campus and by language. And it will be needed in 30 minutes. Or maybe just… “How many conference rooms do we have that don’t have videoconferencing in them at present? Please filter by room occupancy!” Or even just, “How many dual-screen vs. single-screen videoconferencing rooms do we have?”

Remember, this is generic info about the room. We aren’t looking to capture which codec or TV model we have, but we are looking for a broad description. Maybe even a BOM version number that breaks down what’s in the room. What’s the seating capacity? Is there a space to set up food? Is it a divisible space? What other rooms are linked to it?

The more detail you attempt to add at the start, the better prepared you will be down the road. And with that data, the above questions are just a few filters and then you have your report/dashboard. Once you have this accurate info captured and tied to a system that automates the adding and removal of rooms, it sets you up to be able to look at all sorts of things moving forward that can prove invaluable for our users.

As an example, if you have a solid inventory, and an issue arises in a tech space, you can set things up so that when you do an audit and the room fails, you could set a state on the room as offline (totally broken) or let’s say “room diminished” (where maybe the content cable is broken, or a screen won’t roll up), and then push that data to a real-time dashboard for users to see if any rooms that they have meetings in are affected. And even beyond that, depending on other tools you may or may not have in place, you could proactively notify users. And as we move forward and have more and more info at our fingertips, who knows what might happen as we start to see AI tools work their way into our room booking systems.

Inventory management

If the rooms are all in the system, and you have the right tables setup to capture info about the room, now it’s an easy task to start capturing info about the actual gear and linking it to the room/space/location. As in all things here, it’s important to look beyond our basic needs. Sure, we need to capture the standard stuff like make, model and serial number. But to have a better experience down the road, think about not what we can do today, but what might be possible down the road. What about capturing the MAC address? If that is in the system, then it’s not the most difficult thing for a programmer to write a script that can reconcile that MAC address to the IP address and then pull the host name of the device and ensure that based on serial number, they match. Automated audits! Imagine that? This is what we want to get to. How about if we have to swap a codec, and by the time we get back to our desk there is already a ticket telling us that we need to reconcile where the heck that old codec went (since it’s now offline and there is a new device connected to its port with the room name). This is where we put it in a different container/silo. Maybe we have a depot for that campus/building and that’s where it gets placed. Or there is an RMA OUT status that effectively removes it from the systems and links to the new one that comes in. There are lots of possibilities, but you really need to think through not just capital purchases but also the needs of finance and asset management teams.

Another thing that is key when defining equipment is not just the EXACT info about that piece of gear, but generic information. If we wanted to pull a list of every single monitor we have under our umbrella, we would get a list with dozens and dozens, maybe even hundreds, of model names. But if we are pulling that list, odds are good that we want to know two things: We need to know manufacturer (when building relationships with manufacturers) and the size. And as we all know, trying to parse the size of the TV from the model name is absurd. It can’t reliably be done at any kind of scale. So when we capture info about the monitor, let’s also have a drop-down that says 50-59/60-69/70-79/80-89/90+ so that we can quickly see not just numbers by manufacturer, but also size. Add in other filters for install date and maybe replacement date (four years from install?) and now we can clearly see that just via refresh, we will need to buy X number of TVs in size A, B, C over the next year. If you want to make the Sharp/Samsung/LG/NEC, whomever, rep very happy… give them solid numbers. The better you get at capturing critical data like this, the more creating forecasting dashboards is a breeze.

In an optimal, mature, well-planned environment, we would create a CMDB (Configuration Management DataBase) that would capture this info once. And we could have all the inane detail we want to add — power consumption, dimensions, you never know what you need. In reality though, that’s a pretty advanced step for many companies, so you need to set things up in advance with all the info fields that you want and ensure that when you place orders, your your dealers are giving you all that data so that you can easily capture it at the end of a job.

When this really becomes valuable is when you start talking about future forecasting of refreshes and so forth. The more data you have about what was installed when, the easier it is to validate how much money is needed, the easier it is to lock in vendors with better deals and the easier it is for you to see what is coming so that you can get way ahead of the game with blocking off rooms to get that work done.

Getting your ITSM picture to come into focus will require a lot of work. It will require many people thinking about what is important to the needs of their portion of the business. And it will never stop. There will always be some new feature that will allow you to add some new data point, or integrate some other info to expand your dashboards. But if you put the time in, this will all start to come easier and easier.

Creating tickets

Once all the data is in place, then comes the real bread and butter work of the ITSM system: Tickets. A word that management loves to say, a word that front-line folks hate to hear! Tickets are how we show what work has been done, tickets are how we capture metrics, SLA compliance and data to inform insights and outcomes.

In general, there are two types of tickets that get created (I am using ServiceNow terms here as it’s what I am most familiar with, but it’s similar across the board): incidents and catalog items.

In short though, we ticket and track so that we can understand how we are doing compared to published industry norms,and to hopefully see trends emerging that we can head off by adapting what we do.

Incidents

Simple… Something went wrong. It broke, it was affected by some other outage, etc. Incident tickets are important as they need to track a few things. The end the end goal is to always be able to mine them later to come up with actionable items that can, in theory, stop some of those things from happening in the future. Timeline is one of the most important things it captures. When something goes wrong, we are left with a hard record from the time it was first reported, to the time the incident was closed. This can be a network outage that the fault of a service provider, or it can be a component that just happened to die at a bad time. We are left with a record of what happened when and who took what steps at what time. We can track total time and we can determine SLA compliance (Issue ABC we said we could resolve in four hours. And over the last quarter, we resolved it inside that four hours in 96% of all cases.) From there we have actionable data that we can try to improve on. We also can get a clear picture of why that thing kept happening and hopefully the various incident tickets will allow us to extract an action we can take to minimize it from happening in the future. The other great thing is that throughout the IT industry many companies publish/share their metrics and you can see how you compare to peers.

Catalog items

The whole concept of ITSM methodologies is to standardize things to a set of services that are provided, with the purpose being to be able to standardize and deliver consistent results. If anytime a user wanted a new TV, she just went on Amazon and bought one, our ability to support users would plummet. As a result, we create a catalog item to allow users to request that TV. We then purchase one that aligns with all the other needs (that users aren’t even aware of, in most cases) of the business. So the ITSM service catalog is where we have all these items listed.  From the AV (and most of the time, events, as well) side of things, we have a few common items that we have to look at in order to capture the info and data that we need.

Room Evaluation

This is the catalog item you would use internal to the AV team to go check out a room and make sure it’s working OK.  This might be something that is done on a regular cadence, it might only be for certain VIP rooms or it might just be something that is done as needed. Either way, you need a record that is time and date accurate that you went to the room and tested all the functions.

Room Validation

Vendor builds room, vendor hands over room, we test room.

Room breaks, vendor fixes room, vendor hands room back, we test room.

No matter how this transpires, you need a record of what was tested and when. I had a room get fixed once but nobody tested it, so a nonstandard function was not validated, and we all got burned. Testing MUST be done from a checklist and what better way to facilitate that checklist than with your ITSM platform?

Event requests

Events are a big part of our lives. The more details we have at the front, the better off we can meet the user’s needs.  This also allows us the ability to move from service provider to service consultant, helping users plan events even if it’s only by forcing them to capture details of which maybe they hadn’t yet thought.

Equipment requests

“I need a TV for my team!” or maybe “We need the WONKA room to get some new Oompa Loompas,” or whatever.  We need to create a way for users to ask for what they want so that we can facilitate the right upgrades/additions in accordance with standards and processes.

Each of these items has a different audience. Both who requests it, and who ends up with actionable items based on it. For each item, you have to build out both the items you want to capture and also the workflow associated with it.

As a bit of a deeper dive, let’s look at the room audit (or room preventative maintenance, or whatever we want to call it). We need to have a form that has checkboxes/dropdowns for all the functions in the room. And we need to be able to track if it is fine, not fine, or wasn’t fine, but is now fixed.

At a previous employer, we had an issue with touch panels that was easily resolved. Initially the team was reporting as fine. When we added the “fixed on audit” option… all of a sudden, the reported issues numbers skyrocketed and we were able to show that resources needed to be allocated and from there we were able to get to the root and find a fix. But without that option, it was an invisible issue. If an item is left as not fine/broken, then the system would automatically spawn an incident ticket related to broken room and that workflow would take over.

Another reason it is key to have lots of dropdowns and minimal free text is that at the end of the quarter, you might have 5,000+ tickets that you now need to report on. If they are primarily dropdown driven, then a simple pivot table can give you all the data you need. If there is too much text, you have to go through every single one to try to get to the heart of what the issue was. We had a system that was initially created as largely text with some radio button selections. It was a nightmare. And since everyone is going to describe things differently, the tickets really served no purpose other than a record of the total numbers. In a perfect world, we would then lean over to the dozens of developers who have empty plates and are just waiting on a new project and get them to redo everything. Unfortunately, none of us live in that world. So what we ended up doing was just defining a few key phrases and keywords that everyone would use and then all of a sudden, it was far easier to sort/filter the (still text-based) tickets and draw some conclusions.

Drawing conclusions

I told one of my team members (who was driving our efforts to get better info from tickets) was that his goal needed to be that at the end of every quarter, he should be able to go through all the tickets and come up with AT LEAST one actionable item to improve outcomes in some aspect of the service delivery of the team. If you are unable to do that, then you are missing the point. Continuous improvement can only happen with actionable data.

What this tells us is that those tickets that the team all hate… they are your secret weapon to improving outcomes.  The better that data, the more info you can draw out. Make sure that you take into account what is involved with them actually doing the tickets. Anything you invest into getting better, timelier, more accurate data is well worth the investment. iPad to do the audit as you are in the room… YES! QR codes in the room to allow the device to jump right to where it needs to be (no searching for the room name)… YES!

Conclusion

The ITSM system is your ticket to clear insights, solid data and top-notch forecasting. But there is a lot of planning that needs to go into it. The more you think about that future state, the better set you will be to build a strong AV estate.