|
The promise of “big data”—real-time insights, predictive analytics and a better understanding of customer behaviors—has many companies jumping into this near-bottomless information pool with both feet. But the sheer volume of data can cause serious stress for IT professionals trying to balance C-suite expectations, legacy tech limitations and corporate workloads. What’s more, the data market is rapidly evolving. As noted by a recent Bloomberg Businessweek article, big data mainstay Hadoop is now being phased out by power corporations like Google in favor of Flume, Millwheel, Dataflow and a host of other solutions.
Got big data stress? Here are five technologies to help find your zen.
The Internet of Things (IoT)
General Electric calls this the “Industrial Internet,” but it means the same thing—connected devices, everywhere. When every printer, every coffeemaker and every temperature control has a wireless sensor, the amount of big data generated is staggering. And it’s coming soon. Reporting on a keynote speech by CIO Jim Fowler of GE, InformationWeek notes that the company has plans to create “open platforms for ingesting and sharing high-scale data, and building applications while also ensuring security.”
At first glance, the Industrial Internet seems like more work for IT professionals, but as Fowler points out, not everything needs to face outside. For example, while GE has sensor-enabled power plants, these plants aren’t connected to the Internet at large—the result is vast amounts of usable data from a limited sensor set. In other words, while the IoT seems massive, it actually has potential to limit stress.
Visualization
Another key technology to help limit big data stress is visualization. Why? Because data doesn’t naturally form easily understood patterns or spit out simple explanations, meaning it’s often the job of IT departments to communicate data trends to other employees and C-suite execs. Discussion of data without the aid of visual representation makes it nebulous and can lead to skepticism about the veracity of IT predictions. According to Forbes, new visualization tools can “position data insights at the appropriate level of detail,” giving observers an intuitive “snapshot” of critical insights.
Machine Learning (and Human Oversight)
One surefire way to reduce stress caused by big data is to automate the discovery process. As a result, more and more companies now turn to machine learning tools, which not only uncover information as directed but can also discover the unspoken associations between certain types of data, in turn allowing them to find new insights. But these tools have limitations, since algorithms can only go so far before human oversight is required to discover erroneous links or change the direction of inquiry. Here, a combination of autonomous solutions and the occasional human touch squeeze the most from big data.
Cloud-Based Storage
Aside from the sheer amount of space needed, companies also need to consider their storage method. As noted by The Conversation, SQL has been the de facto standard for decades, but doesn’t do well with unstructured data—such as social media relationships, indexed Web documents or digital video collections. The evolution of “NoSQL” technologies like Document Store, Wide Column Store, Search Engine, Key-Value Store and Graph Database combined with a large-volume cloud space can help alleviate store-and-search stressors.
Network Partitioning
It’s also critical to consider your network. Big data processes typically require “bursts” of processing power owing to the sheer number of small calculations running in parallel, meaning it’s a good idea to create multiple, separate compute spaces. The cloud is an ideal network-partitioning environment since it allows for both logical and physical abstraction in addition to on-demand scalability.
Want to lower your big data stress? Think IoT, visualization and machine learning at the local level—along with cloud-based storage and networking to really calm your nerves.
Sponsored byWhoisXML API
Sponsored byCSC
Sponsored byRadix
Sponsored byVerisign
Sponsored byDNIB.com
Sponsored byVerisign
Sponsored byIPv4.Global
The relevant big data encompasses the IoT and IoY (Internet of You) for human bodies & wearables. The objective is to capture data & control the environment - Cisco is predicting that 50 billion “things” could be connected to communications networks within six years, up from around 10 billion mobile phones and PCs today.
The four elements:
1. Hardware
(a) Computing characterized by small, often dumb computers attached to objects. These devices sense and transmit data about the environment or offer new means of controlling it. Researchers are designing computers the size of a pinhead to collect data.
(b) A wireless cellular network exclusively for things. The French company SigFox says it has picked the San Francisco Bay Area to demonstrate such a network.
(c) Wireless devices
(d) Powered by small batteries
2. Software
(a) Data capture and control
(b) Data visualization
(c) Analytics: New generation of machine learning: “Robots That Learn Through Repetition, Not Programming” (see http://bit.ly/1shxuLk).
3. Storage devices
4. Strategy shift: Need for internal integration of software, hardware and analytics.