What do you mean by “Big Data”? How is it rel­e­vant to Dig­i­tal Marketing?

Over the last year, there has been a lot of buzz regard­ing “Big Data” ana­lyt­i­cal tools.  There is an excel­lent arti­cle by McK­in­sey - Big data: The next fron­tier for inno­va­tion, com­pe­ti­tion, and pro­duc­tiv­ity that sum­ma­rizes the oppor­tu­nity and chal­lenge of big data. There is no clear tech­ni­cal def­i­n­i­tion of what big data is, but the best frame­work to think about big data is in terms of four V’s.

  1. Vol­ume: The vol­ume of data col­lected by dig­i­tal mar­keters is upwards of 10’s of TB.  Any and every solu­tion needs to scale eco­nom­i­cally to query and ana­lyze the data in a rea­son­able amount of time
  2.   Veloc­ity:  Most of dig­i­tal mar­ket­ing data that comes in is real-time i.e click stream and trans­ac­tional data.  Ide­ally, the solu­tion should make the data avail­able for analy­sis in real-time.
  3. Vari­ety: Dig­i­tal Mar­keters are inter­ested in get­ting data from var­i­ous chan­nels i.e web, call cen­ter, social and trans­ac­tional data (more infor­ma­tion in my pre­vi­ous blog).  Sup­port­ing var­i­ous types of data becomes crit­i­cal for dig­i­tal mar­ket­ing analysis.
  4.  Visu­al­iza­tion: Query­ing large multi-dimensional data sets is always a chal­lenge, and SQL does not make it easy for dig­i­tal mar­keters.  The sum­ma­rized results from queries can be large too.  Sim­ple visu­al­iza­tions can be an ele­gant method to query and ana­lyze the data.

How does Adobe’s Insight address “Big Data”

Insight is a hor­i­zon­tally scal­able solu­tion which processes large vol­umes of data in real-time.  The­o­ret­i­cally, at a com­puter sci­ence model level, it scales sim­i­larly to Hadoop.  As the vol­ume of data increases, one can add more machines to the Insight clus­ter to opti­mize the query per­for­mance.  The solu­tion is cost-effective, as there is no pro­pri­etary hard­ware involved.

Insight pri­mar­ily uses visu­al­iza­tions to query and ana­lyze data.  The visu­al­iza­tions make Insight a pow­er­ful tool to do sequen­tial event-based analy­sis. Insight pro­vides an incred­i­bly sim­ple mech­a­nism to seg­ment cus­tomers based on behav­ior, attrib­utes or attrib­utes of cus­tomer data.  The seg­men­ta­tion capa­bil­ity dri­ves mar­keters to quickly inden­tify poten­tial growth/profitable cus­tomer segments.

I would like to add few sen­tences to dif­fer­en­ti­ate between a report­ing and an ana­lyt­i­cal tool.  A report­ing tool presents the met­rics (based on the data) with lim­ited abil­ity to slice the data by its attrib­utes.  An ana­lyt­i­cal engine gives the abil­ity to deter­mine why a par­tic­u­lar met­ric is higher or lower.  An ana­lyt­i­cal engine gives dig­i­tal mar­keters the abil­ity to test or ver­ify your mar­ket­ing hypoth­e­sis. Using an ana­lyt­i­cal engine is about spend­ing hours/days explor­ing the data to find the next nugget of knowl­edge that can pro­pel your cam­paign or your company.

Over the years, it has been great fun to see many com­pa­nies build large cus­tomer ana­lyt­ics solu­tions using Insight to address dif­fi­cult mar­ket­ing prob­lems. The beauty of the solu­tion lies in its sim­ple con­cepts from com­puter sci­ence, and yet it is a pow­er­ful solu­tion to solve dif­fi­cult ana­lyt­i­cal problems.

In my next blog, I am going to give a spe­cific exam­ple of how a large finan­cial ser­vices com­pany has used Insight to increase cross-sell.