1-Workspace Flow

Proper workspace area delineation and reports management is a key project component. Discover users can break up analysis areas into defined named workspaces with organized groups of reports. Single reports can be copied or moved across workspaces by right clicking within a report tab. If a user right clicks on a workspace tab there are three options to ‘Rename Workspace’, ‘Close Workspace’, or ‘Duplicate Workspace’. The ‘Duplicate Workspace’ option can replicate all existing report build work in a fresh workspace to make adjustments or apply different segment filters. Discover lends best to an exploratory approach and users gain efficiency advantages working through data investigations in this same manner.

Dis­cover Workflow Tips 1.1: Workspace Right Click Actions

The second workspace option is to create a new workspace slate that is at first empty without any defined reports by clicking the + tab to the right of any active workspaces. A great number of reports can exist within a single workspace, but it quickly can reach the point where not all the reports can be visible in a workspace at once which requires user scroll efforts to reach a desired report. Therefore the recommendation is to leverage the ability to create multiple workspaces and work through various reports that hold the same analysis intent focus.

Dis­cover Workflow Tips 1.2: Workspace Right Click Actions

2-Folder Management

The menu folder customization options are features of Discover that exist with the Metrics and Segments Windows. The ‘Organize Metrics’ option is selected by clicking on the small wrench icon within the Metrics left side menu panel.  The ‘Organize Metrics’ window allows for the building of grouped metrics along whatever associations a user prefers. Given the vast amount of variations on a per metric basis that open up with Discover, the ‘Organize Metrics’ option can save sorting and search time to pull in frequently used metrics within a profile. While the standard non-editable categories of metrics will always exist and cannot be altered, the option to customize metrics and calculated metrics within grouped folders is a critical organization feature.

Dis­cover Workflow Tips 1.3: Organize Metrics Menu Option

Dis­cover Workflow Tips 1.5: Organize Metrics Menu Window

The next aspect of Discover folders is the ‘Organize Segments’ option which can be accessed by clicking on the small wrench icon within the Segments left side menu panel. The ‘Organize Segments’ menu allows for collections of segments to be logically grouped and set together. By default there is a ‘Favorites’ folder outside of the locked ‘Suite Segments’ and ‘Pre-Configured’ folders. It is recommended users organize segments on a functional basis, or on a per project basis to keep segments accessible and in order. The powerful option to share Discover segments across an entire company is also available as an optional check box option within the ‘Organize Segments’ window.

Dis­cover Workflow Tips 1.6: Organize Segments Menu Option

Dis­cover Workflow Tips 1.7: Organize Segments Menu Window

3– Left Menu Window Pane Customization & Report Hide Chart Option

Another aspect of deep customization is the layout of the entire left side Discover menu. The Default View (1) lays out the ‘Dimensions’, ‘Metrics’, ‘Segments’, and ‘Granularities’/‘Date Ranges’ in descending order. The layout can be wholly customized to only include the relevant analysis components to a project. For example a collapsed view which provides more coverage for the main values list can be customized via moving the windows into a Nested View (2). The Customized View (3) option completely hides specific layout menu content panes so that only the relevant values are present. The power within this feature is over the course of work users can shift their menu options to best meet their immediate needs in pulling data elements. Users can always revert back at any time to the Default View (1) by selecting the Window>Default View option from the top menu line in Discover.

Dis­cover Workflow Tips 1.8: Left Menu Window View: Three Customization Option [(1) Default View, (2) Nested View, (3) Customized View]

A simple action which can greatly improve the visible data set range is to hide the main default Chart from a given Discover report. Users can click and drag the four dot selector beneath a chart upward and then the entire chart will be minimized which expands the amount of raw data values that are shown in the main table. Selection of the same four dot selector and dragging down will bring the chart visibly back. Often users in Discover want to bridge data value sets and export to Excel; if this is the case then hiding the Chart makes getting a better initial read on the data set easier.

Dis­cover Workflow Tips 1.9: Step 1– Chart Hide Option User Action

Dis­cover Workflow Tips 2.0: Step 2– Data Table Only View (Chart Now Hidden)

4– Trend Cell Right Click Action

The trend cell quick right click action allows for a single defined time range to be built into a new isolated trended report per user granularity date parameters. To initiate a user must right click a metric cell within a report, then select the ‘Trend Cell menu option.  Next choose a defined date granularity setting. A new trended report is then automatically built and opened within the same workspace for user review. It is one of the best deep-dive on demand features in Discover because it provides fast side exploration of data trends and anomalies without disrupting the current report data view.

Dis­cover Workflow Tips 2.1: Trend Cell Menu Option

5– Column Properties Right Click Action

The next within report right click action is the second menu item ‘Column Properties’ which alters the formatting of a given metrics column in terms of values shown and visible blue bar graph indicators. These options allow for users to do immediate switches and augmentation of raw metric counts to % of total values. The ‘Column Properties’ can save users time through holistic identification of metric drops and spikes in conjunction with the raw data counts for the period.

Dis­cover Workflow Tips 2.2: Column Properties Menu Option

Dis­cover Workflow Tips 2.3: Column Properties Menu

Conclusion

The Discover menu pane customization options and quick run actions can prove to be huge time savers over the course of analysis work. Users would be well served to keep these options in their back pocket as go-to methods of adjustments during analysis efforts. It bears emphasizing how an organized and coherent Discover project can benefit in repeatable work, work transitions to other users, and established work approach. Discover opens up a wide range of task flow capabilities and can bend to where users need to the tool to focus both short and long term.

Brian Au is a con­sul­tant in Adobe Con­sult­ing, focused on dig­i­tal strat­egy, ana­lyt­ics & opti­miza­tion for retail & travel clients. He tweets at @BrianAu.

So how then do you arm yourself to maximize outcomes and to combat poor data discipline? The key is in understanding key concepts of statistics, so that you can find when and how promises are made that cannot possibly be true. You do not need to understand the equations, or even have masterly level depth on most of these topics, but it is vital that you understand the truth behind certain types of statistical claims. I want to break down the top few that you will hear, and how they are misused to make promises, and how do you really achieve that level of success.

Correlation does not Equal Causation –

Problem- I don’t think anyone can get through college without having heard this phrase, and most can quote it immediately, but very few really focus on what it means. The key thing to take from this is that no matter how great your correlative analysis is it can not tell you cause of the outcome nor the value of items without direct active interaction with the data. No matter how much you can prove a linear correlation or even find a micro-conversion that you believe is success, by itself it can never answer even the most basic of real world business questions. They can be guiding lights towards a new revelation, but they can also just be empty noise leading you away from vital information. It is impossible to tell if you leave the analysis at just basic correlation, yet in almost all cases this is where people are more then happy to leave their analysis. The key is to make sure that you do not jump to conclusions and that you incorporate other pieces of information instead of blindly following the data.

Just because I can prove a perfect correlation between email sign-ups and conversion rate, that they both go up, I can never know from correlation alone if getting more people to sign-up for emails CAUSED more conversions, or if the people we got to convert more are also more interested in signing up for email. In a test this is vital because not only is it easy see those two points, but you are also limited with only a single data point making even correlation impossible to diagnose. It is incredibly common for people to claim they know the direction and that they need to generate more email signups in order to produce more revenue, but it is impossible to make that conclusion based on purely correlative information alone and it can be massively damaging to a business to point resources in a direction that can equally produce negative and not positive results.

The fundamental key is to make sure that you are incorporating consistent ACTIVE interaction with data, where you induce change across a wide variety of items and measure the casual value of them. Combined or leading your correlative information you can discover amazing new lessons that you would never have learned before. Without doing this the data that many claim is leading them to conclusions is often incomplete for fundamentally wrong and can in no way produce the insights that people are claiming. The core goal is always to minimize the cost of this active interaction with data while maximizing the number and level of alternatives that you are comparing. Failure to do this will inevitably lead to lost revenue and often false directions for entire product road maps as people leverage data to confirm their opinions and not to truly use data rationally to produce amazing results.

Examples – Multiple success metrics, Attribution, Tracking Clicks, Personas, Clustering

Solution – Causal changes can arm you with the added information needed to answer these questions more directly, but in reality that is not always going to be an option. If nothing else, always remember that for any data to tell you what lead to something else, you have to prove three things:

1) That what you saw was not just a random outcome

3) That you need to prove causal direction to be able to prove any conclusion

Just the very act of stopping people from not racing ahead or abusing this data to prove their own agenda will dramatically improve the efficiency of your data usage as well as the value derived from your entire data organization.

Rate vs. Value –

Problem – There is nothing more common than finding patterns and anomalies in your analytics. This probably is the single core skill of all analysis, yet it can often be the most misused or abuse actions taken with data. It can be segments that have different purchase behavior, channels that behave differently, or even “problems” with certain pages or processes. Finding a pattern or anomaly at best is simply the halfway point of actionable insight, not the final stop to be followed blindly. Rate is the pattern of behavior, usually expressed as a ratio of actions. Finding rates of action is the single most common and core action in the world of analytics, but the issue usually comes when we confuse the pattern we observe with the action to “correct” that action. Like Correlation vs. Causation above though, a pattern by itself is just noise. It takes active interaction and comparison with other less identified able options in order to validate the value of those types of analysis.

Just because Google users spend 4.34 min per visit or email users average visit depth is 3.4 pages are examples of rates of action. What this is not is the measure of value of those actions. Value is the change in outcome created by that certain action not the rate at which people happen to do things in the past. Most people understand “past performance does not ensure future outcomes” but they fail to apply the same logic when it comes to looking for patterns in their own data. Value is expressed as a lift or differentiation, things like adding a button increased conversion by 14% or removing our hero image generated 18% more revenue per visitor.

The main issues come from confusing the ability to measure different actions with knowing how to change someone’s behavior. The simplest example of this is the simple null hypothesis of what would happen if that item wasn’t there? Just because 34% of people click on your hero image which is by far the highest amount on your homepage, what would happen if that image wasn’t there? You wouldn’t just lose 34% of people, they would instead interact with other part of the page. Would you make more less revenue? Would it be better or worse?

It also comes down to two different business questions. At face value the only possible question you could answer with just pattern analysis is, “What is an action we can take?”, in the ideal value business case you would instead answer “Based on my current finite resources, what is the action I can take to generate the most X” where X is your single success metric. Rates of value have no measure of ability to change or of cost to do so, and as such they can not answer many of the business questions that they are erroneously applied to.

Examples – Personalization, Funnel Analysis, Attribution, Page Analysis, Pathing, Channel Analysis

Solution – The real key is to make sure that built into any plans of optimization you are incorporating active data acquisition and a that you are always measuring null assumptions and measuring the value of items. This information combined with knowledge of influence and cost to change can be vital, but without it is likely empty noise. There are entire studies in math dedicated to this, with the most common being bandit based problem solving. Once you have actively acquired knowledge, you then will start to build information that can start to inform and improve the cost of data acquisition, but never replace it.

These are but two of the many areas where people consistently make mistakes when leveraging data and concepts from statistics to make false conclusions. Data should be your greatest asset not your greatest liability, but until you help your organization make data driven decisions and not data validated decision there are always going to be massive opportunities for improvement. Make it a focus to improve your organizations understanding and interaction with each of these concepts and you will start using far less resources and making far better outcomes. Failure to do so also insures the opposite outcomes over time.

Understand data and data discipline have to become your biggest areas of focus and educating others your primary directive if you truly want to see your organization take the next step. Don’t let just reporting data or making claims of analysis be enough for you and you will quickly find that it is not enough for others.

Every day, customers leave a trail of digital bread crumbs. From computers and notebooks to smartphones and in-store transactions, the information contained in available data streamlines the customer experience. It localizes and personalizes it. If your company isn’t gobbling up that trail, using it to gain market share advantage, you aren’t paying attention.

The cool thing about the whole digital marketing process is that data can be analyzed and used in ways that are helpful to both your customer and your company. It saves the customer time and money. It directs them to your product. It provides you with ways to improve branding, generate demand, and influence decisions at critical points.

Starting with something as simple as a friend’s 140-character comment, a filtered photo, a mobile ad, or a drive-by billboard, and ending with a localized offer or directions to a point of sale, the path to purchase can be complex. That’s where a bespoke analytics program, tailored to your goals, is so effective.

Many marketing departments do not understand the methodology behind applying analytics to their marketing plans. They don’t pay attention. They nudge new strategies aside with a wince of apprehension. They fear the result. They’re still rooted to print, which by the way, continues to have strong support, but only in some niche markets. If you aren’t paying attention to what your data is telling you, you’re losing an edge that someone else will use to cut into your market share.

Change can be hard. When you shine the light of analytics on the performance of your marketing plans, media buys, branding, and sales, some previously murky things become transparent. Some long-held love affairs with old ways may prove ineffective. Some strategies may hold pleasant surprises, proving much more profitable than ever imagined. You won’t know unless you pay attention. And do the math. Just to make sure you’re doing it right, ask yourself, and your peers and employees, these questions:

Are your marketing goals clearly defined? Your goals and objectives drive your media choices. If not clearly stated, you may be sending resources to the wrong channels. Misappropriated marketing dollars are painful, and anything that does not positively reinforce brand perception and brand awareness is just plain ugly. Put analytics to good use here and direct dollars to performance.

Are those choices backed up with accurate, empirically proven attribution via analytics? Analytics can help you here, but only if you are spotting the stars and ditching the dogs, based on solid metrics. Connecting the attribution dots between sales, orders, leads, and return on investment to the numbers on searches, emails, displays, PR, and even offline channels in print, radio, and TV will put marketing on a credible, accountable, solid status. Data in, analytics applied, attributable results out. No shadows. No magic. Just facts that illustrate and emphasize the importance of marketing as a profit center and sales driver.

Then, you will have everyone’s attention. Even that of your competitors.

Negative correlations are generally used to spot anomalous indicators. A negative correlation means that as one of two values increases, the other decreases. This means that as one value becomes bigger, the other proportionately shrinks. For instance, let’s look at your common office printer. Every office manager knows that as you print more pages, the amount of toner decreases. So you have one piece of the puzzle growing in number while the other decreases―a negative correlation. How much of a negative correlation has yet to be defined though. There are other variables such as color versus black ink and the number of the pages.

Both positive and negative correlations can look a lot like cause and effect, but they are not. Causality is a direct action to a direct result. If I kick a football, it flies away from my foot; that is cause and effect. There’s no room for interpretation. Now, it could be said that there are a ton more data related to the event (and there are), but the action and outcome are directly cause and effect.

And here’s where we get to the ambiguous subtitle. Recently while using a trending tool available online, I looked up two seemingly unrelated topics: knitting and poison. It became immediately obvious there was a negative correlation between the two. The searches for knitting tapered off as the searches for poison increased at almost exactly the same rate. Either both crafty folks and grandmothers were giving up knitting in favor of poisoning their neighbors (with elderberry wine) or there was some other underlying factor. With a family reunion on the horizon, I had no other choice but to investigate further―it was a matter of survival, you see. Luckily, I work where I do, which means I have some tools available to me for just this sort of problem.

As an analyst working with “big data,” correlating metrics is an exploratory process that helps you discover unknown relationships. The data (or dimensions) can be date, visitor ID, geography, or any number of other applicable dimensions. Ultimately, you can correlate against any dimension type, but from a best-practice standpoint and keeping within the theory of statistics, you need at least 30 observations, instances, or values. Keeping this in mind, I took on the mystery of why Nana was putting down her knitting needles and buying strychnine. I plugged my variables into Adobe Analytics Premium. The color-coded correlation matrix gave me visual points of reference, and I was able to come up with an answer efficiently.

How efficiently is the question? The table below has 16 metrics based on data from a Web service. Remembering our “30 rule,” these metrics were trended over 35 days. Which metrics have the strongest positive correlation with “visits”? It is almost impossible to tell with the naked eye. Our brains are just not designed to absorb all this data in the manner it is presented, at least not when we’re evaluating numerical values on the scale “big data” requires.

However, when we open up a correlation matrix, we can see both numerically and with a heated color map. This makes it easier to identify the metrics with the strongest correlation. Our brain is able to make the connection much more quickly than when we’re looking at just three colors (black, white, and one shade of blue above). As it turns out, visits and single access have the strongest positive correlation (r = 0.983) in this matrix.

So back to Granny and our knitting/poison mystery. As it turns out, the time of year was the factor. During the winter months, when people are indoors more, knitting was more prominently searched. When the weather began to warm up, the searches for poison increased, but the instances for knitting searches dropped off. Likely, as pests and weeds began to take hold, people started looking for ways to eradicate them. Conversely, since the weather was warmer, there was less interest in knitting sweaters and blankets,.

Once I was satisfied that this was indeed the cause, I packed for the reunion, secure that I could again enjoy Aunt Marie’s green beans wrapped in bacon. If I had based my decision as to whether or not to go to the reunion solely off the initial data provided, I would have opted out of my family’s event entirely.

Understanding diagnostic analytics and being able to correlate (both positive and negative) correctly provides an invaluable source of information, but it’s not the only tool to help ferret out anomalous data. Next week we’ll look at Anscome’s Quartet and show how a visual representation of values can help you more accurately identify anomalies before they can become impactful. Watch out for “Anscome’s Quartet,” or “Not All Numbers Were Created Equal.”

Everyone wants credit for the onsite conversion, but which channels should persist, expire or be ignored? Should some channel click-throughs get prioritized over other channels? Marketing Channels includes a feature to allow marketers to override certain channels. But what are overrides and how do they work?

Overrides are available in the Marketing Channel Manager when you first set up your channel names. By default every channel except Direct and Session Refresh has a check mark next to it. But what does that check mark actually do?

To answer that question let’s focus on last touch allocation (more to come on allocation in the next post). When accounting for 100% of all traffic entries (which the Marketing Channels report does), every time someone enters the site that entry is allocated to one channel or another. So every time a Paid, Organic or Direct click-through to the site occurs, it is being recorded as the visitor’s “Last Touch” channel. In this scenario the most recent channel always over-writes the previous channel.

Before the Marketing Channels report was included in SiteCatalyst, Campaigns were silo-ed into their own report so marketers would use an expiration period on the campaign to persist the tracking code until the campaign expired or the visitor clicked-through on another campaign. That means if a visitor clicked on a Paid Search Ad to the site, but converted on a subsequent visit from an organic or direct channel (within the expiration period) the conversion would be attributed back to that original Paid Search Ad. SEM and Paid Media teams loved this model, but SEO, Social Media and earned efforts were essentially ignored in the report. With Marketing Channels, organic channels can be on par with paid channels. Which means Natural Search can override Paid Search as a “Last Touch” channel, and Social Media can override Display, etc. Even Direct and Session Refresh can prevail over other channels, which is why SiteCatalyst includes the ability to determine which Overrides are available.

Maybe you don’t want Direct to overrule Paid Search. And we certainly don’t want Session Refresh to override any other channels. In this case you can elect Direct and Session Refresh (and any other channels) to be ignored, which would allow Visits and Success Metrics to be allocated back to the previous channel the visitor clicked through on.

Before we dive into examples, we have one more notion to sort out…Visits. The Visits metric in the Marketing Channels report is a conversion metric. That means that Visits will behave like other success metrics in regards to allocation. Let’s say I have 500 Visits to my site. In the Marketing Channels report I can see which channel each of those visits entered the site on; I can sum up all the visits from each channel and come up with 500. Yes! I finally have my channel Pie Chart for Visits!! If I let all channels override all other channels equally, then my Visit channel will always be the most recent channel clicked-through by the visitor (Click-throughs = Visits). But what happens if some channels are not allowed to override? Overrides essentially mean the channel will be ignored.

If Direct is not allowed to override (unchecked in the Marketing Channel Manager), that means that if I have a recognized channel already recorded in a previous visit, the Direct click-through will be ignored and the visit and success events can be allocated back to the previous channel. For example, a site has a visitor with 2 visits: the 1st from Paid Search, the 2nd from Direct and the visitor converts during that second visit. The conversion will be tied back to that Paid Search entry rather than the Direct entry. And since Visits are a conversion metric the Visit from the 2^nd entry will also be allocated back to Paid Search.

4 Visitors in a world where all channels Override (everything checked):

Same 4 Visitors when Direct and Session Refresh are not allowed to Override other channels:

How does this change the game? Well, when making a decision on whether to use Overrides consider the following:

• Advantages: Conversions can be allocated to “true” channels
  • Option: Use Click-throughs as the denominator and understand “x Visits and y conversions resulted from z click-throughs” (not y conversions resulted from x Visits)
• Disadvantages: Report can be confusing and Direct may be ignored as a “branding” channel for other offline efforts
  • Alternative: Do not use Overrides for Direct and use First Touch Marketing Channel reports to allocate conversions back to the Original channel for acquisition analysis