UNIT 6 RECENT TRENDS

Big data Big data is a buzzword, or catch-phrase, used to describe a massive volume of both structured and unstructured data that is so large it is difficult to process using traditional database andsoftware techniques. In most enterprise scenarios the volume of data is too big or it moves too fast or it exceeds current processing capacity. Despite these problems, big data has the potential to help companies improve operations and make faster, more intelligent decisions.

Is Big Data a Volume or a Technology? While the term may seem to reference the volume of data, that isn't always the case. The term big data, especially when used by vendors, may refer to the technology (which includes tools and processes) that an organization requires to handle the large amounts of data and storage facilities. The term big data is believed to have originated with Web search companies who needed to query very large distributed aggregations of loosely-structured data.

An Example of Big Data An example of big data might be petabytes (1,024 terabytes) or exabytes (1,024 petabytes) of data consisting of billions to trillions of records of millions of people—all from different sources (e.g. Web, sales, customer contact center, social media, mobile data and so on). The data is typically loosely structured data that is often incomplete and inaccessible.

Hive Hive is a data warehouse infrastructure tool to process structured data in Hadoop. It resides on top of Hadoop to summarize Big Data, and makes querying and analyzing easy.

Apache Pig Apache Pig is a tool used to analyze large amounts of data by represeting them as data flows. Using the PigLatin scripting language operations like ETL (Extract, Transform and Load), adhoc data anlaysis and iterative processing can be easily achieved. Pig is an abstraction over MapReduce. In other words, all Pig scripts internally are converted into Map and Reduce tasks to get the task done. Pig was built to make programming MapReduce applications easier. Before Pig, Java was the only way to process the data stored on HDFS. Pig was first built in Yahoo! and later became a top level Apache project. In this series of we will walk through the different features of pig using a sample dataset. LINK TO STUDY IN DETAIL http://www.rohitmenon.com/index.php/apache-pig-tutorial-part-1/

Real-Time Business Intelligence Real-time business intelligence is an approach to data analytics that enables business users to get up-to-the-minute data by directly accessing operational systems or feeding business transactions into a real-time data warehouse and business intelligence (BI) system. 

Real-Time Business Intelligence In today’s competitive environment with high consumer expectation, decisions that are based on the most current data available will improve customer relationships, increase revenue, and maximize operational efficiencies. The speed of today’s processing systems has moved classical data warehousing into the realm of real-time. The result is real-time business intelligence (RTBI). Business transactions are fed as they occur to a real-time business intelligence system that maintains the current state of the enterprise. The RTBI system not only supports the classical strategic functions of data warehousing for deriving information and knowledge from past enterprise activity, but it also provides real-time tactical support to drive enterprise actions that react to immediate events. As such, it replaces both the classical data warehouse and the enterprise application integration (EAI) functions.

operational business intelligence Operational business intelligence, sometimes called real-time business intelligence, is an approach to data analysis that enables decisions based on the real-time data companies generate and use on a day-to-day basis. Typically, the data is queried from within an organization’s enterprise applications.  Operational business intelligence technology is primarily targeted at front-line workers, such as call center operators, who need timely data to do their jobs.

Agile Business Intelligence Agile business intelligence addresses a broad need to enable flexibility by accelerating the time it takes to deliver value with BI projects. It can include technology deployment options such as self-service BI, cloud-based BI, and data discovery dashboards that allow users to begin working with data more rapidly and adjust to changing needs. To transform traditional BI project development to fit dynamic user requirements, many organizations implement formal methodologies that utilize agile software development techniques and tools to accelerate development, testing, and deployment. Ongoing scoping, rapid iterationsthat deliver working components, evolving requirements, scrum sessions, frequent and thoroughtesting, and business/development communication are important facets of a formal agile approach.

Embedded BI Embedded BI (business intelligence) is the integration of self-service BI tools into commonly used business applications. BI tools support an enhanced user experiencewith visualization, real-time analytics and interactive reporting. A dashboard may be provided within the application to display relevant data, or various charts, graphs and reports may be generated for immediate review. Some forms of embedded BI extend functionality to mobile devices to ensure a distributed workforce can have access to identical business intelligence for collaborative efforts in real time.

Cloud Business Intelligence Cloud business intelligence (cloud BI) refers to network-based tools that turn raw data into information that businesses can use to cut costs, streamline inefficiencies, increase revenue and generally make better organizational decisions. Cloud-based BI can perform just about any business intelligence function: Data visualization Process mining Data mining Text mining Online analytical processing (OLAP) Querying Business performance management Benchmarking Statistical analysis Forecasting Reporting