Entries Tagged 'Rethinking IT' ↓

The licensing dispute between Fox and Cablevision which kept the Giants game from coursing across the Cablevision network at kickoff today shows us yet again why closed licensing models based upon licensing aggregation simply are not in the best interest of consumers.  As I highlighted in a previous post, I’m not a fan of having my cable company providing content contingent upon my letting them negotiate on my behalf without my input.  By restricting my choices, by forcing itself as an unwanted middleman, it holds me hostage as a bargaining chip.

It’s not uncommon within traditional media, and it’s being forced upon new forms of media consumption and distribution.  Proprietary methods of licensing, securing, and distributing content seek to retain control, while purporting to solve the problem of enabling fair compensation for copyright holders.

Large media companies are in the business of exploiting economies of scale to monetize content.  Aggregating customer demand and simplifying its own job of authorizing access to content by providing as uniform a product as possible, cable companies both remove the market choice for content, subsidizing what it provides at its own discretion, and with limited provider choice in most markets, provides service that’s "good enough".

I remain convinced that what will ultimately destroy large media companies will be the disintermediation of forced relicensing schemes by the establishment of an open content authorization standard, and service providers which compete on the basis of the quality of their service instead of the exclusivity of their agreements.  An open content authorization standard, complete with near-real-time analytics, support for cryptographically-secure delegation of authorization (to allow for voluntary relicensing schemes), and an ecosystem of disinterested authorization providers can enable a new transparency in the media supply chain.

If Greenpeace can force Timberland into transparency about the sources of its leather purchased from Brazilian suppliers, why is the relative exploitation of both producers and consumers of copyrighted materials free of such transparency when it’s so incredibly simple to track the provenance of digital assets?  Should I be able to filter the content I’ll pay for based upon the business practices of the company providing it?

This past week, I listened to Tim Armstrong, CEO of AOL, giving a keynote speech about the future of his company.  Two things stuck with me from what he spoke about — one being my introduction to the AOL property Seed, the other being his description of what was needed to create "offramps from the internet", programming.  Seed is a site for writers and photographers to create freelance content from which AOL can draw, while compensating contributors.  His reference to programming was to evoke the notion of television or radio programming, what most new media types refer to as content curation.

AOL’s building its businesses and brands around content curation, it’s doing so by sourcing its inputs directly from independent contractors, placing thousands or millions of tiny bets in a marketplace that can’t be controlled by its competitors.

An open, transparent authorization system could still be used to allow exclusive licensing access to a single customer, should the copyright licensor choose, but an open platform provides the opportunity for new revenue models based upon ubiquitous access, low transaction costs (at high volume), and simplifying consumption.  Authorization policies can allow new methods of consumption, entitlement to value-added content, and clarity on policies for derived works.  One can easily imagine a "cover" of a popular song inheriting obligations from the original, but allowing a new performance by an amazing coffeehouse talent to distribute her performance in ways that still ensure that the songwriter’s licensing restrictions (say, a royalty of tenth of a cent per play) are respected.  Content curators, such as those doing the "programming" would have an incentive to add value.  Content distributors would have an incentive to compete on the value offered by their distribution service, and not on their ability to negotiate licensing deals.

People can run from the comfort of Comcast/Cablevision/Time Warner/Verizon/DirectTV/Dish to the comfort of iTunes, but they’re simply exchanging one master for another.  Eventually, even Netflix will face its day of reckoning — its model still relies upon large-scale aggregate content deals it makes on behalf of its customers.

Eventually, open WILL win.  Market forces will eventually dictate it.  Want to get ahead of the curve and help make it happen?

I do.

Continuing the Rethinking IT series, let’s take a look at data.

If one of my underlying principles is that today’s IT is yesterday’s data processing (plus communications), then data’s been fairly well acknowledged to be old hat for IT professionals, right?

We’re doing it wrong
Take a look at your document management system (that’s if you have one of those, and then only if it’s actually used) and see how much of your company’s operating data set is stored, unstructured, in Microsoft Office formats.

How well is it maintained? How does it get shared? How do updates get communicated? Are updates propagated to operational systems? Shouldn’t they be, via some workflow?

Does the status of a routine workflow require meetings to update a project manager? If so, you’re probably doing it wrong. Some of the most important data processing the IT department should be doing is processing data about what it’s doing. Activity logging is essential to knowing how your systems are operating. Are you monitoring your systems from an enterprise monitoring solution that hits a URL or runs a synthetic transaction periodically to determine system health? If you’re polling every 5 minutes, is that really good enough? All it tells you is that the system responds sufficiently to monitoring requests that occur periodically. If you’re responsible for operating a system, you shouldn’t simply be checking for the negative (failure to respond to polling requests), but validating the positive.

Data-Driven Dashboards
We need them at all levels.

Someone “on the ground” who’s going to respond to system problems must be able to positively identify how the system is performing in order to determine whether or not a problem has been corrected. I can no longer accept, “It must be working if nobody’s complaining,” and neither should you. Your front-line guys need “operational intelligence”.

Someone who’s responsible for overseeing a routine process (one for which there’s a defined workflow) should be able to see the status of that workflow and work with individuals to resolve problems, without having representatives of all groups in the workflow attending status meetings. If I had a gigabyte for every time I’ve had to “get on the same page” for a routine, predictable, data-driven process, I could… well, I could store a lot.

Of course, as it gets up the chain, the dashboard will get more focused around business KPIs, and less about technology, but it’s the collection, correlation, and analysis of business activity data that bubbles up into those metrics.

Dr. Dobb’s has a great article about greenshifting. Short and sweet of it? Greenshift is how the “color” assigned to the status of a project gets greener as it approaches the CIO. If your organization assigns colors based upon anything other than observable metrics, you’ll never know who to trust.

Which one do you mean?

IT is full of overloaded terms, and business users bring their own vocabularies. An established corporate taxonomy is a must, as well as the basis for defining the schemata of your core business entities. If you can’t agree on the fundamentals, you can’t interoperate. As well, unique instance identifiers must be used globally to associate additional data from disparate sources. Once you’ve established an instance identifier and some relationships, additional data can be joined from key-value stores or services emulating them, with data maintained by different business units responsible for correlated data. Key-value stores can scale extraordinarily well, and lend themselves naturally to data partitioning.

If we need to exchange data, we MUST agree on a way to communicate “which one” of “which thing” unambiguously. Any text field which can be edited by a non-sysadmin DOES NOT work. If your identifier is “what you call it”, you’re doing it wrong, because “what you call it” can change, and your data must maintain its relationships despite a regime change in the naming department.

Envelope backs ARE important
Metrics are useless if they’re not sanity-checked.

If you’re doing what’s essentially key-value storage, if you know the average size of the objects you’ll store and multiply out by the number of expected objects, you’ll find that effective caching can keep most of your working set in memory for data that are mostly read. Even if you’re using an Oracle RDBMS for storage, database ninjas can do wonders with gobs of RAM. A Dell server with 8 cores and 256GB of RAM starts at about $21k before other hardware and software options, but that pales in comparison with the software license and support. Load up on RAM, and use it effectively.

If you’re storing user preferences for a million users, and the preferences objects will be at most 1k in size, then before accounting for overhead, you’re talking about a gigabyte of data. If you use that type of calculation as your starting point, it cuts through quite a layer of irrelevant crud. Sanity check against fundamentals. Allow for a large margin of safety if you’re required to spec anything out early, since you’ll always have been optimistic about overhead, or someone will have found another function that’ll require your resources, since they’re “available”.

Good, fast, and cheap ARE all possible
The economics of IT have fundamentally shifted in significant ways. Software and software support costs can make up the majority of the costs of maintaining a cheap, powerful server.

Open source alternatives are available for many of the commercial products you use. Commercially-supported “respins” of “Community Editions” can offer significant savings — and potentially better support for your use cases. Creative systems architecture folks can help you assess how you can meet the appropriate SLAs for your services.

To do this effectively, however, they’ll need that with which we started this discussion. Data.

Data about availability requirements. Data about access requirements. Data about access patterns. Data about consistency or transaction support requirements.

Recap for the impatient who scroll to the end to get to the point
Data’s got to be explicitly identifiable. IT activity data’s got to be stored and analyzed both in near-real-time as operational intelligence and for long-term trend analysis, “live-data replays” for accurate performance testing, and for resource and requirements planning. Proper data-driven dashboards at all levels indicate how things are working, as opposed to “no news is good news.”

I am consistently finding myself challenging assumptions about the role of IT and information systems in business. I’m also finding that I’ve not been able to consolidate my ideas in a way that provides the value of aggregation and integration, and I’ve decided that thinking my way to a comprehensive belief system and approach on this blog is as good a place as any.

Perspective
The most fundamental asset in IT planning, as well as one of the most important deliverables from IT to its customers including itself, is perspective.

Information
The most valuable asset in gaining perspective is information. Information, as distinguished from data, is the product of analysis of data.

Service
Service is about providing information — whether interactive views of real-time data sources, analytics of user behavior, usage reporting, billing data, user authorization information — in a usable format, reliably and predictably. Service orientation isn’t just about SOAP and Web Services. It’s about providing access to retrieve, store, or modify information and data, and providing a service level agreement for that access.

Data
Data is the heart of IT. Before it was called Information Technology, it was called Data Processing. Collection, storage, management, performance, and availability of data are essential to generate the information required to run IT and provide value to the larger business. Core business entity data, metadata (data about data), service and access data (logging), and authorization data are core business assets and are required to provide information and services and perspective. Effective structuring of business data should be considered a core responsibility of the IT department.

Having laid out some of my principles (and I welcome comments and discussion), I’m looking forward to the next post in this series, where I will start to look at the value chain within IT.