Technical requirements of big data


2.     What are the technical requirements of big data?


This data boom presents a massive opportunity to find new efficiencies, detect previously unseen patterns and increase levels of service to citizens, but Big Data analytics can’t exist in a vacuum. Because of the enormous quantities of data involved in these solutions, they must incorporate a robust infrastructure for storage, processing and networking, in addition to analytics software.


1.     Storage Solutions

Often, organizations already possess enough storage in-house to support a Big Data initiative. (After all, the data that will be processed and analyzed via a Big Data solution is already living somewhere.) However, agencies may decide to invest in storage solutions that are optimized for Big Data. While not necessary for all Big Data deployments, flash storage is especially attractive due to its performance advantages and high availability.

 

2. Processing Power

 

Servers intended for Big Data analytics must have enough processing power to support this application. Some analytics vendors, such as Splunk, offer cloud processing options, which can be especially attractive to agencies that experience seasonal peaks. If an agency has quarterly filing deadlines, for example, that organization might securely spin up on-demand processing power in the cloud to process the wave of data that comes in around those dates, while relying on on-premises processing resources to handle the steadier, day-to-day demands.

 

3. Analytics Software

 

Agencies must select Big Data analytics products based not only on what functions the software can complete, but also on factors such as data security and ease of use. One popular function of Big Data analytics software is predictive analytics — the analysis of current data to make predictions about the future. Predictive analytics are already used across a number of fields, including actuarial science, marketing and financial services. Government applications include fraud detection, capacity planning and child protection, with some child welfare agencies using the technology to flag high-risk cases.

 

4. Networking Hardware

 

The massive quantities of information that must be shuttled back and forth in a Big Data initiative require robust networking hardware. Many organizations are already operating with networking hardware that facilitates 10-gigabit connections, and may have to make only minor modifications — such as the installation of new ports — to accommodate a Big Data initiative. Securing network transports is an essential step in any upgrade, especially for traffic that crosses network boundaries.

.

.

.

https://fedtechmagazine.com/article/2016/12/4-infrastructure-requirements-any-big-data-initiative

Comments

Post a Comment

Popular posts from this blog

Limitations of traditional data analysis

Image
 7 .      What are the limitations of traditional data analysis? Traditional statistics is about analysing and summarizing data, and suited to processing lesser amounts of linear, repeatable data to find a solution. It’s very useful in environments where data, and the relationships between the data, are relatively stable. It is still by far the most commonly used methodology and continues to be good for mid to long term forecasting based on prior sales history, where a few hundred or even fewer data points may generate a reasonable forecast. The important limitations of statistics are:   Statistics laws are true on average. Statistics are aggregates of facts, so a single observation is not a statistic. Statistics deal with groups and aggregates only. Statistical methods are best applicable to quantitative data. Statistics cannot be applied to heterogeneous data. If sufficient care is not exercised in collecting, analysing and interpreting the data, statistical results might be misleadi
Read more

Traditional data analysis

Image
 6 .      What is Traditional data analysis (descriptive and inferential)?          Traditional data is the structured data which is being majorly maintained by all types of businesses starting from very small to big organizations. In traditional database system a centralized database architecture used to store and maintain the data in a fixed format or fields in a file. For managing and accessing the data Structured Query Language (SQL) is used. 3 Characteristic of Traditional Data: ·        Traditional analytics is static.   Traditional data analytics typically relies on dashboards composed of visualizations. These dashboards are based on common business questions and are predefined well in advance.   ·        Traditional analytics answers “what.”   Traditional analytics answers a series of “what” questions and leaves the user to determine “why” through their own analyses.   ·        Traditional analytics is driven by hypotheses.   Dashboards are typically predefined based on common
Read more