One of the most talked-about changes from Site­Cat­a­lyst 14 to Site­Cat­a­lyst 15 is the way data is processed and how long it takes until activ­ity on the site is vis­i­ble in the report­ing — the so called “latency”.

Site­Cat­a­lyst col­lects data from web sites or apps. It then processes the data and cre­ates a mul­ti­tude of reports, all read­ily avail­able in the Site­Cat­a­lyst user inter­face. Data flows in and gets aggre­gated so it makes sense.

Our engi­neer­ing teams have made sig­nif­i­cant changes in Site­Cat­a­lyst 15, mean­ing Site­Cat­a­lyst 14 and Site­cat­a­lyst 15 process data in an entirely dif­fer­ent way.

Let’s look at two myths that I hear a lot:

Site­Cat­a­lyst 14 is real-time”

That is part of the truth, but not the com­plete picture.

Data pro­cess­ing in Site­Cat­a­lyst fol­lows the prin­ci­ple “process as soon as you can”.

When a vis­i­tor calls up a web page, her browser sends a track­ing request. Site­Cat­a­lyst will almost imme­di­ately count this as a Page View. The visit, how­ever, is counted once it’s fin­ished, mean­ing after 30 min­utes of inactivity.

As a result, some of the reports in Site­Cat­a­lyst are prac­ti­cally real-time, the most pop­u­lar exam­ple being the Site Con­tent > Pages report. Cus­tom Traf­fic reports also dis­play real-time data for the Page Views metric.

Other reports are at least 30 min­utes behind, because the sys­tem will not process visit-based met­rics before the visit has actu­ally timed out. The same goes for some of the con­ver­sion reports and metrics.

Site­Cat­a­lyst 15 has a latency of 2 hours”

That is not accurate.

Site­Cat­a­lyst 15 processes data in batches. Those batches are usu­ally 60 min­utes of col­lected data. As an exam­ple, all track­ing data that arrives into Site­Cat­a­lyst between 11am and 11:59:59am is part of the same batch.

At 12:00, Site­Cat­a­lyst starts col­lect­ing data into a new batch. At the same time, the “11am batch” goes into pro­cess­ing. Pro­cess­ing of a 1-hour batch cur­rently takes around 30 minutes.

So, when do we see data?

If a track­ing request came in at 11:00:01am, it’ll be part of the “11am batch” which will fin­ish pro­cess­ing at around 12.30. The data will there­fore be vis­i­ble just under 90 min­utes after it happened.

If, on the other hand, the track­ing request came in at 11:59:59am, it’ll also be part of the “11am batch” and there­fore also vis­i­ble at 12.30, a mere 30 min­utes after it happened.

As a result of the batch approach, latency looks like this:

[Screenshot]

Latency in Site­Cat­a­lyst 15 — 60 ver­sus 30 minute slices

Notes

Not all data is processed in 60 minute batches. The batch size can be changed to 30 min­utes for crit­i­cal report suites (2 per customer).

Because a 30 minute batch can be processed in roughly 15 min­utes, the max­i­mum time for a hit to appear in Site­Cat­a­lyst is 45 min­utes, and the short­est time is 15 min­utes, fol­low­ing the same saw-tooth pat­tern as above.

What if you are fol­low­ing web activ­ity for a new release or after hav­ing sent a newslet­ter? What if your edi­to­r­ial team needs to know right now how that new arti­cle about that politi­cian performs?

Site­Cat­a­lyst 15 now has “Cur­rent Data” reports which work a lot like Site­Cat­a­lyst 14 worked: each report in Site­Cat­a­lyst 15 has a “Cur­rent Data” coun­ter­part which pro­vides the same latency that Site­cat­a­lyst 14 did.

[Screenshot]

Cur­rent Data Reports in Site­Cat­a­lyst 15

Those reports can be used to check data fo the cur­rent day or cur­rent day and the day before.

If you need Cur­rent Data reports, go to Admin > User Man­age­ment and add a user to the “Cur­rent Data” group. That user will now be able to see those reports.

[Screenshot]

Cur­rent Data Reports Group