The big data challenge can only be adequately addressed when knowledge of various different fields such as data mining, machine learning algorithms, parallel processing, and data management practices are effectively combined.

Scientific disciplines face a huge challenge due to large amounts of date. Hence, ‘smart data analytics methods’ are required to deal with these data sets.

There are ‘six concrete phases’ that are typical for a data mining project: (1) problem/business understanding; (2) data understanding; (3) data preparation; (4) modeling; (5) evaluation; (6) deployment.

·  ‘problem/business understanding phase’ – the general steps such as stating the objectives and goals of the situation(e.g. data analytics success criteria)

·  ‘data understanding’-a few basic reports are made for the data given initially (collection and data sets), including data (collection) description, exploration options, and data quality statements

·  ‘data preparation’- refers to the tasks the precisely elaborate the data sets in the problem, and whether they should be included or not . It also reformats or deletes the data as required

·   ‘modeling’- includes the selection of modeling techniques, text designs and related parameter settings to build the model

·  ‘evaluation’- the steps which assess the results and make sense out of it

·   ‘deployment’- enables a systematic way of performing analytics on a regular basis including a monitoring and maintenance plan.

The concept offers a solid foundation and the setup is plausible, however multiple algorithms are required to interpret the problem at hand. Methods of smart data analysis allows for using petabytes rather than gigabytes or terabytes as used in the past. This makes it efficient for large data sets, to get information out of them.

The central issue related to the topic is, that this is entirely against the desire of VPH (Virtual Physiological Human) Researchers to build detailed mathematically simplified mechanistic models for each individual patient. But what seems prudent at the current stage is to adopt a middle path between the two; that is: use a model, but make it partly phenomenological and partly mechanistic.

Thus, this paper proposes that we can successfully adopt this ‘middle path’ and combine Big Data Analytics successfully with VPH Technologies to yield viable medical solutions. For this to be implemented, it is mandatory that we further develop existing Big Data technologies to cope with certain specific applications that arise due to the merging of two mildly incompatible fields.

The paper further outlines that there is a crucial need for diversion of research grants to this ‘hybrid’ field of medicine, or at least better target the research funds into this field, for the simple reason that this field has many complex requirements and technical issues. Such requirements are: working with sensitive data; analytics of complex and mixed up data spaces, including non-textual information; distributed data management under constraints of security and performance; specialized analytics to integrate bioinformatics and systems biology information with clinical observations at tissue, organ and organisms scales; and specialized analytics to define the ‘physiological envelope’ during the daily life of each patient.

In conclusion, we see that Big Data Technologies have immense potential in the field of Biomedicine, but the only stumbling block in its immediate implementation appears to be the occurrence of a large number of technical snags. It remains necessary to operate with an open mindset so as to continue domain related research, but also to not repeat past mistakes and take calculated risks with the new Big Data Technology.

In data mining, the old databases of individuals or groups that are collected are analysed, visualised, categorised and summarised to generate new databases of that individual or group. This article discusses the techniques of data mining with big data. Big data is a repertoire of large, complex data that are from autonomous, multiple sources.

Data mining can be done using various methodologies such as Neural Network, Decision Trees, Genetic Algorithm, Rule Extraction etc.

Artificial neural network deals with the proper analysis of biological neural patterns by simulating the connections that exist between sets of data in an actual biological neural network. By performing such analysis, new databases that are compatible with the neural pattern and are extensions of old databases can be generated.

Decision tree are used to simplify complex decisions into smaller logical decisions which mostly contain two outcomes. This allows effective simulation of human reaction in such situations.

In Genetic algorithms method, new databases are made based on the family background and history. Here, the analysis of the individual or organisation are made up to the level of their chromosomes. This helps in detection of non mutated databases that can be produced from the traits of individual. Decisions, actions that are made by individuals which contradict their traits cannot be executed properly.

With the advancement of technology day by day , accuracy of data prediction has been increasing which helps in analysing even non traited decisions, actions etc. of the individuals.

Are there specific reasons for this lag? What are they? What are the improvements in technique that have resulted from the use of this technology in healthcare?

 One of the reasons health care is lagging behind other industries is it has relied for too long on standard regression-based methods that have their limits. Many other industries, notably retail, have long been leveraging newer methods such as machine learning andgraph analytics  to gain new insights. But health care is catching up.

For example, hospitals are starting to use graph analytics to evaluate the relationship across many complex variables such as laboratory results, nursing notes, patient family history, diagnoses, medications, and patient surveys to identify patients who may be at risk of an adverse outcome. Better knowledge and efficient assessment of disparate facts about patients at risk could mean the difference between timely intervention and a missed window for treatment.

1. https://www.youtube.com/watch?v=2IZtPhVC5L0

2. A technical paper from IEEE.

3. The doubt was answered via extensive Googling

4. Some books from BPHC library:

http://172.16.100.176:8080/dspace/

5. Future of Big Data:

http://www.ibm.com/analytics/us/en/industry/healthcare/

6. Current status of world's opinion in big data:

http://www.hisa.org.au/bigdata/

7. Why healthcare is one of the only ways out:

https://hbr.org/2014/12/why-health-care-may-finally-be-ready-for-big-data

Q1: Does big data mining help students choose a career?

A. Yes

B. No

C. Can't Say

Q2. Do you think big data mining can play a major role in improving the lives of the people in the near future?

A. Yes

B. No

C. To some extent

D. Can't say

Q3. What type of technologies support big data ?

A. Storage

B. Compute

C. Interaction

D. All

Q4. Will cloud computing be cost effective for modernizing statistical products and processes ?

A. Yes

B. No

C. Can't say

Q5. What are the challenges in implementing big data in a company ?

A. Security

B. Budget

C. Lack of talent

D. Enterprise not ready for big data

1. A technical paper from IEEE

2. For basic undderstanding

http://www-01.ibm.com/software/data/infosphere/hadoop/what-is-big-data-analytics.html

3. Some books from the digital library of Bits Pilani Hyderabad http://172.16.100.176:8080/dspace/

4. Techniques of data mining http://www.thearling.com/text/dmtechniques/dmtechniques.htm

5. Applications of data mining from http://www.tutorialspoint.com/data_mining/dm_applications_trends.htm

6. A part of journal for the use of data mining in business environment http://www.tutorialspoint.com/data_mining/dm_applications_trends.htm

7. Some statistics related to data mining fromhttp://onlinelibrary.wiley.com/journal/10.1002/(ISSN)1932-1872

Financial Data Analysis

The financial data in banking and financial industry is generally reliable and of high quality which facilitates systematic data analysis and data mining. Some of the typical cases are as follows −

• Design and construction of data warehouses for multidimensional data analysis and data mining.

• Loan payment prediction and customer credit policy analysis.

• Classification and clustering of customers for targeted marketing.

• Detection of money laundering and other financial crimes.

Retail Industry

Data Mining has its great application in Retail Industry because it collects large amount of data from on sales, customer purchasing history, goods transportation, consumption and services. It is natural that the quantity of data collected will continue to expand rapidly because of the increasing ease, availability and popularity of the web.

Data mining in retail industry helps in identifying customer buying patterns and trends that lead to improved quality of customer service and good customer retention and satisfaction. Here is the list of examples of data mining in the retail industry −

• Design and Construction of data warehouses based on the benefits of data mining.

• Multidimensional analysis of sales, customers, products, time and region.

• Analysis of effectiveness of sales campaigns.

• Customer Retention.

• Product recommendation and cross-referencing of items.

Telecommunication Industry

Today the telecommunication industry is one of the most emerging industries providing various services such as fax, pager, cellular phone, internet messenger, images, e-mail, web data transmission, etc. Due to the development of new computer and communication technologies, the telecommunication industry is rapidly expanding. This is the reason why data mining is become very important to help and understand the business.

Data mining in telecommunication industry helps in identifying the telecommunication patterns, catch fraudulent activities, make better use of resource, and improve quality of service. Here is the list of examples for which data mining improves telecommunication services −

• Multidimensional Analysis of Telecommunication data.

• Fraudulent pattern analysis.

• Identification of unusual patterns.

• Multidimensional association and sequential patterns analysis.

• Mobile Telecommunication services.

• Use of visualization tools in telecommunication data analysis.

Biological Data Analysis

In recent times, we have seen a tremendous growth in the field of biology such as genomics, proteomics, functional Genomics and biomedical research. Biological data mining is a very important part of Bioinformatics. Following are the aspects in which data mining contributes for biological data analysis −

• Semantic integration of heterogeneous, distributed genomic and proteomic databases.

• Alignment, indexing, similarity search and comparative analysis multiple nucleotide sequences.

• Discovery of structural patterns and analysis of genetic networks and protein pathways.

• Association and path analysis.

• Visualization tools in genetic data analysis.

Other Scientific Applications

The applications discussed above tend to handle relatively small and homogeneous data sets for which the statistical techniques are appropriate. Huge amount of data have been collected from scientific domains such as geosciences, astronomy, etc. A large amount of data sets is being generated because of the fast numerical simulations in various fields such as climate and ecosystem modeling, chemical engineering, fluid dynamics, etc. Following are the applications of data mining in the field of Scientific Applications −

• Data Warehouses and data preprocessing.
• Graph-based mining.
• Visualization and domain specific knowledge.

Intrusion Detection

Intrusion refers to any kind of action that threatens integrity, confidentiality, or the availability of network resources. In this world of connectivity, security has become the major issue. With increased usage of internet and availability of the tools and tricks for intruding and attacking network prompted intrusion detection to become a critical component of network administration. Here is the list of areas in which data mining technology may be applied for intrusion detection −

• Development of data mining algorithm for intrusion detection.

• Association and correlation analysis, aggregation to help select and build discriminating attributes.

• Analysis of Stream data.

• Distributed data mining.

• Visualization and query tools.

The article talks about using smart data analytics for large datasets. Regarding this several questions arise.

Is there a limit on the date we can analyze? Can this date be analyzed in an efficient manner? What are the ways to interpret this data? What are the practical applications for big data analytics?

The limit for the data that can be analyzed keeps growing every day. It is only through the smart data analytics methods that are discussed that the data can be analyzed in an efficient and judicious manner.

The data is only interpreted when knowledge of multiple disciplines such as data mining, machine learning algorithms, parallel processing, and data management practices are effectively combined.

The applications are manifold and spread across multiple spheres. Understanding climate change, finding alternative energy sources, and preserving the health of an ageing population are all cross-disciplinary problems that require high-performance data storage, smart analytics, transmission and mining to solve.

2. Some books from the digital library of Bits Pilani Hyderabadhttp://172.16.100.176:8080/dspace/

3. To understand the recent trends pertaining to the article

http://www.computerworld.com/article/2690856/big-data/8-big-trends-in-big-data-analytics.html

4. Using smart data analytics in manufacturing systems

http://www.nist.gov/el/msid/lifecycle/rtdasms.cfm

5. Research Data Alliance (RDA), Big Data Analytics (BDA) interest group, Online: https://rd-alliance.org/internal-groups/bigdata-analytics-ig.html

6. Marbán Ó., Mariscal G., Segovia J., ‘A Data Mining & Knowledge Discovery Process Model’, Book Data Mining and Knowledge Discovery in Real Life Applications, ISBN 978-3-902613-53-0, pp. 438-453, 2009

7. Provost F. and Fawcett T., ‘Data Science and its relationship to Big Data and Data-Driven Decision Making’, doi: 10.1089/big.2013.1508, Big Data, Vol.1(1), 2013

8. Provost F. and Fawcett T., ‘Data Science and its relationship to Big Data and Data-Driven Decision Making’, doi: 10.1089/big.2013.1508, Big Data, Vol.1(1), 2013

Students and teachers interested in learning about smart data analytics. Big companies who can use the methods discussed to optimize their work.

Q1: Does the term ‘smart data analytics interest you at all?

A.Yes

B.No

C.Can't Say

Q2.Do you feel smart data analytics can practically improve the lives of people around the world?

A.Yes

B.No

C.To some extent

D.Can't say

Q3. Do you feel companies around the world would implement ‘smart data analytics’?

A.Yes

B.No

C.To some extent

D.Can't say

Q4. What could be the possible hindrances to a company implementing ‘smart data analytics’?

A.Not enough efficiency

B.Lack of funds

C.Not enough human resources

D.Lacking the need for ‘smart data analytics’

E. Other

Q5. What is now your basic understanding of ‘smart data analytics’?

The survey can help in gauging the view of students, teachers and companies towards the topic. It will help in highlighting the problems and the demand for smart data analytics. In this way we can improve to eliminate the flaws and also understand the feasibility of using smart data analytics in a practical scenario.

Survey:

1. Will Big Data be able to bring a noticeable change to the healthcare industry?

a. No

b. Yes

c. Can't say

2. Do you think it might have grave implications in the long run to be so tech dependent?

a, Yes

b. No

c. Can't say

3. While data collection can be implemented, do you think practical application of this much data in analysis is possible/ feasible?

a. Yes

b. No

c. Can't say

4. Do you think collection of this much data is feasible?

a. Yes

b. No

c. Can't say

5. Would you agree to this being implemented in your nearest hospital?

a. Yes

b. No

c. Don't care

Concretization- to make concrete, specific, or definite.

Anomaly- something that deviates from what is standard, normal, or expected.

Metadata- a set of data that describes and gives information about other data.

Algorithm- a process or set of rules to be followed in calculations or other problem-solving operations, especially by a computer.

Outlier- a person or thing differing from all other members of a particular group or set.

Cloud architecture is highly scalable and allows Netflix to quickly provision computing resources as its sees the need. Traffic patterns are analysed across device types and localities to help improve the reliability of video streaming and plan for growth.

The technology is also used for Netflix’s recommendation engine based on a customer’s viewing habits and stated preferences.

Pricing

Sticking with Netflix, the Washington Post theorises that Netflix could vary its price if it had enough information on each user to know how much they might pay. 

To a certain degree, this happens in online retail with airlines targeting previous browsers, and some stores (such as Staples) changing prices depending on which physical store the customer is nearest.

The Wall Street journal has also documented that Orbitz, the travel website, has in the past charged Mac visitors higher than those on Windows. Taking into account IP, device, age, past visits, and more variables, throwing them into a database and calculating a charging threshold can conceivably be termed big data.

Retail Habits

Target uses a wealth of customer data to predict future purchasing habits. Specifically, pregnancy kicks off a chain of purchases that are fairly distinctive – Target’s data collection is spookily prescient, sending one teen customer nappy vouchers before her own father knew she was pregnant.

Weather

WeatherSignal works by repurposing the sensors in Android devices to map atmospheric readings. Handsets such as the Samsun S4, contain a barometer, hygrometer (humidity), ambient thermometer and lightmeter. 

Obviously, the prospect of millions of personal weather stations feeding into one machine that will average out readings is exciting, and one that has the potential to improve forecasting. 

Heart Disease

IBM are predicting heart disease with big data. Analysis of electronic health record data could reveal symptoms at earlier stages than previously.

IBM uses the Apache Unstructured Information Management Architecture (UIMA) to extract the known signs and symptoms of heart failure from available text. 

With no single strong indicator, only weak signals or ‘co-morbidities’, such as hypertension, diabetes, associated medications, ECG and genomic data etc. can be analysed. Drawing out probabilities from disparate and size-differing databases is a task for big data analytics. 

Infectious diseases

Again IBM, looks at a model and data from the World Health Organization. IBM looked at local climate and temperature to find correlations with how malaria spreads. This analysis is used to predict the location of future outbreaks. The Spatio Temporal Epidemiological Modeler (STEM) is free and open source.

Doctor performance

Crimson is a system that shows variables including complications, hospital readmissions and measures of cost. It colour codes signals as to how well a doctor is performing against his or her peers.

The Wall Street Journal suggests the technology has reduced average stay and average cost at the Long Beach Memorial Hospital. 

Neural Network: artificial neural network is a biological system that detects patterns and makes predictions

Generic: characteristic of or relating to a class or group of things; not specific

Semantics: It is the study of meaning. It focuses on the relation between signifiers, like words, phrases, signs, and symbols, and what they stand for; their denotation

Fidelity: cause, or belief, demonstrated by continuing loyalty and support.