Home / Blogs

Big Data Stress? Go Zen With These 5 Technologies

The promise of “big data”—real-time insights, predictive analytics and a better understanding of customer behaviors—has many companies jumping into this near-bottomless information pool with both feet. But the sheer volume of data can cause serious stress for IT professionals trying to balance C-suite expectations, legacy tech limitations and corporate workloads. What’s more, the data market is rapidly evolving. As noted by a recent Bloomberg Businessweek article, big data mainstay Hadoop is now being phased out by power corporations like Google in favor of Flume, Millwheel, Dataflow and a host of other solutions.

Got big data stress? Here are five technologies to help find your zen.

The Internet of Things (IoT)

General Electric calls this the “Industrial Internet,” but it means the same thing—connected devices, everywhere. When every printer, every coffeemaker and every temperature control has a wireless sensor, the amount of big data generated is staggering. And it’s coming soon. Reporting on a keynote speech by CIO Jim Fowler of GE, InformationWeek notes that the company has plans to create “open platforms for ingesting and sharing high-scale data, and building applications while also ensuring security.”

At first glance, the Industrial Internet seems like more work for IT professionals, but as Fowler points out, not everything needs to face outside. For example, while GE has sensor-enabled power plants, these plants aren’t connected to the Internet at large—the result is vast amounts of usable data from a limited sensor set. In other words, while the IoT seems massive, it actually has potential to limit stress.


Another key technology to help limit big data stress is visualization. Why? Because data doesn’t naturally form easily understood patterns or spit out simple explanations, meaning it’s often the job of IT departments to communicate data trends to other employees and C-suite execs. Discussion of data without the aid of visual representation makes it nebulous and can lead to skepticism about the veracity of IT predictions. According to Forbes, new visualization tools can “position data insights at the appropriate level of detail,” giving observers an intuitive “snapshot” of critical insights.

Machine Learning (and Human Oversight)

One surefire way to reduce stress caused by big data is to automate the discovery process. As a result, more and more companies now turn to machine learning tools, which not only uncover information as directed but can also discover the unspoken associations between certain types of data, in turn allowing them to find new insights. But these tools have limitations, since algorithms can only go so far before human oversight is required to discover erroneous links or change the direction of inquiry. Here, a combination of autonomous solutions and the occasional human touch squeeze the most from big data.

Cloud-Based Storage

Aside from the sheer amount of space needed, companies also need to consider their storage method. As noted by The Conversation, SQL has been the de facto standard for decades, but doesn’t do well with unstructured data—such as social media relationships, indexed Web documents or digital video collections. The evolution of “NoSQL” technologies like Document Store, Wide Column Store, Search Engine, Key-Value Store and Graph Database combined with a large-volume cloud space can help alleviate store-and-search stressors.

Network Partitioning

It’s also critical to consider your network. Big data processes typically require “bursts” of processing power owing to the sheer number of small calculations running in parallel, meaning it’s a good idea to create multiple, separate compute spaces. The cloud is an ideal network-partitioning environment since it allows for both logical and physical abstraction in addition to on-demand scalability.

Want to lower your big data stress? Think IoT, visualization and machine learning at the local level—along with cloud-based storage and networking to really calm your nerves.

By David Eisner, President & CEO at Dataprise, Inc

He founded Dataprise in 1995 and has led its growth from tiny start-up to recognized leader in providing managed IT services to small and medium-size businesses.

Visit Page

Filed Under


I basically regroup the elements of the post. Alex Tajirian  –  Sep 24, 2014 8:37 AM

The relevant big data encompasses the IoT and IoY (Internet of You) for human bodies & wearables. The objective is to capture data & control the environment - Cisco is predicting that 50 billion “things” could be connected to communications networks within six years, up from around 10 billion mobile phones and PCs today.

The four elements:
1.  Hardware
(a)  Computing characterized by small, often dumb computers attached to objects. These devices sense and transmit data about the environment or offer new means of controlling it. Researchers are designing computers the size of a pinhead to collect data.
(b)  A wireless cellular network exclusively for things. The French company SigFox says it has picked the San Francisco Bay Area to demonstrate such a network.
(c)  Wireless devices
(d)  Powered by small batteries

2.  Software
(a)    Data capture and control
(b)  Data visualization
(c)  Analytics: New generation of machine learning: “Robots That Learn Through Repetition, Not Programming” (see http://bit.ly/1shxuLk).

3.    Storage devices

4.    Strategy shift: Need for internal integration of software, hardware and analytics.

Comment Title:

  Notify me of follow-up comments

We encourage you to post comments and engage in discussions that advance this post through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can report it using the link at the end of each comment. Views expressed in the comments do not represent those of CircleID. For more information on our comment policy, see Codes of Conduct.

CircleID Newsletter The Weekly Wrap

More and more professionals are choosing to publish critical posts on CircleID from all corners of the Internet industry. If you find it hard to keep up daily, consider subscribing to our weekly digest. We will provide you a convenient summary report once a week sent directly to your inbox. It's a quick and easy read.

I make a point of reading CircleID. There is no getting around the utility of knowing what thoughtful people are thinking and saying about our industry.

Co-designer of the TCP/IP Protocols & the Architecture of the Internet



IPv4 Markets

Sponsored byIPv4.Global

New TLDs

Sponsored byRadix


Sponsored byDNIB.com

Threat Intelligence

Sponsored byWhoisXML API

Domain Names

Sponsored byVerisign


Sponsored byVerisign

Brand Protection

Sponsored byCSC