ZEMA Eases Data Set Shaping
Analysts in energy and commodity markets are expected to assess the status of particular markets in a timely fashion to stay on top of ever-changing conditions and trends. In order to do so, they must aggregate (analyze) and compare data from various data sources, in different granularities. In other words, analysts need to have “data dexterity”, meaning they must have tools and techniques that equip them to work with and analyze large volumes of data.
Shaping formulas are one of the most common techniques used by analysts to identify market patterns. Data set shaping can reveal a quick view of the relation between data series in a particular interval. However, applying shaping formula to data series is a daunting task, as disparate data series must be standardized before calculations or mapping can be performed. The process of manually standardizing data prior to processing it often leaves analysts little time to perform advanced analysis. For example, in MS Excel, dealing with shaping formulas for large datasets with a high granularity requires programming and a long processing time. By contrast, those who employ ZEMA can use shaping formulas easily and efficiently.
The first functionality that must be employed in ZEMA prior to applying a shaping formula to data is aggregation. ZEMA’s standard aggregation functionality allows analysts to perform data aggregation for many data sources in any time granularity (such as hourly, daily, weekly, monthly, quarterly, and annually) while the solution’s cumulative aggregation functionality allows users to calculate the value of data within a specified time interval. Next, shaping formulas can be applied. Some ZEMA shaping formulas include shape average, shape maximum, shape minimum, and shape median. With these formulas, analysts can easily filter their data to focus on a desired period of time, while using other data to generate reference lines. Shaping formulas allow analysts to determine the historical shape of one data set and compare it with current data to discover market trends. This comparison can be performed across different markets.
Figure 1: Shaping Average of Electricity Prices (AESO) vs. Temperature (AccuWeather) – April 13, 2014
Figure 1 is an example of the application of ZEMA’s shaping formula within the power market: the graph displays two separate data series: hourly price and average temperature. In this example, AESO hourly electricity prices are displayed against Calgary’s hourly temperature on April 13, 2014. The actual hourly price of the day in AESO (represented by the orange bar above) is similar to the hourly actual temperature (the blue line). To investigate a pattern between the two data series with more certainty, let’s look at a broader time frame.
Figure 2: Hourly Average of Electricity Prices (AESO) vs. Temperature (AccuWeather) – April 1-30, 2014
In Figure 2, the hourly shaping price average between April 1, 2014 and April 30, 2014 (green line) is sketched against the hourly shaping temperature average (red line) in the same period. The correlation between the two dotted lines is 0.70, which shows a close relation between hourly electricity prices and the temperature fluctuations in April 2014. The correlation derived from this monthly data can be used to justify the price movement on April 13th. Actual daily price is affected by many factors, including temperature, generation, and market demand. The impact that any one of these factors has upon price changes due to seasonality as well. Shaping formulas, though, are able to extract the hidden relation between different factors easily and accurately.
To learn more about how ZEMA can help with your data collection, analysis, or integration, book a complimentary demo. It may surprise you to know what can be achieved through clean data and powerful analytics!https://blog.ze.com/the-zema-solution/zema-eases-data-set-shaping/https://blog.ze.com/wp-content/uploads/2013/11/iStock_000000330791Large-1024x682.jpghttps://blog.ze.com/wp-content/uploads/2013/11/iStock_000000330791Large-300x300.jpgThe ZEMA SolutionAccuWeather,Accuweather Daily Temprature,Accuweather Monthly Temprature,AESO,electricity prices,Shaping AverageAnalysts in energy and commodity markets are expected to assess the status of particular markets in a timely fashion to stay on top of ever-changing conditions and trends. In order to do so, they must aggregate (analyze) and compare data from various data sources, in different granularities. In other...Winson FangWinson Fangwinson.firstname.lastname@example.orgContributorBlogs by data management Experts & Analysts | ZE