I hope you will consider adding caching details to RUM reporting.
- Share of cacheable and non-cacheable resources
- Cached vs uncached resources
- Resource download size? How else would I know how performance is? Timing alone is not enough, is it?
- Improved resources reporting, enabling exploration of requests and headers
- Use filtering/segments such as device type. There's a Device Type report, but it's very basic
- Page groups and filtering
- Recordings, maybe? Snapshots are near useless
- Javascript warnings?
In general I'd say this:
Site24x7 RUM should enable us to pinpoint areas of improvement, issues and report on improvements. It doesn't live up to that.
Adding a bit more to the feature request by quoting blogs.akamai.com/2015/12/cdns-performance-and-monitoring---part-2.html
Now let us look at the KPIs from your front end (RUM) monitoring:
- Performance affected by various factors - load time across various dimensions
- By page - how long each web page took to render on the browser
- By mobile devices - on mobile you have less bandwidth, are pages smaller?
- By various browsers
- By geography
- By Edge cache hit or cache miss (might require integration with your CDN metrics)
- Error frequency and types of errors, by page
- Performance of various steps in the browser's functionality
- Download time - how long did it take the browser to suck in all the content
- DOM time - once data is sucked in, how long to parse and build the parse tree
- Render time - once all the data was read in and parsed, how long to actually render it
- Size of content - ensure pages are not too heavy
Quoting another site:
Benefits of RUM
RUM can help identify how page visitors are using a site or app. It also provides page metrics such as load time for analysis of a page, objects and even individual visits.
- Measure service level targets easily. It offers real-world measurement of key targets by tracking actual visits and delivering top-level data on actual use cases.
- Easily identify problems and better prioritize issues. It can replay user sessions and track transaction paths to surface hidden problems.
- Determine hitches at the network and page level. Problems at the lower levels of a website can hide like needles in a haystack. It can spotlight these problems even when they’re intermittent in nature or based on rare conditions.
Hi Allan,
Thanks for your detailed briefing regarding your requirements.
We would like to convey a few points regarding the feasibility of the requested
> "Share of cacheable and non-cacheable resources" and "Cached vs uncached resources"
It is not possible to capture these in a straight forward manner. This information is available from the headers of the request and it is not possible to fetch the headers of already loaded pages or resources unless we make another request and capture the headers. Nevertheless, if there is a way to capture these, we will definitely add this to our roadmap and update the same here.
>Resource download size? How else would I know how performance is? Timing alone is not enough, is it?
We are working on adding the resources download size and page size along with the response times and we will update the same here once it goes live.
>Improved resources reporting, enabling exploration of requests and headers
It is not possible to report all the resource requests as there will be hundreds of different resources loaded in an application and hence these are filtered by domains and shown in the Resources tab. As we have mentioned earlier it is not straightforwardly possible to fetch request headers of the loaded resources but we will look into the possibility of capturing these.
>Use filtering/segments such as device type. There's a Device Type report, but it's very basic, Page groups and filtering
It will be helpful if you could elaborate on your requirements regarding this. If you are asking about the webpages filtering by location, device and browser, we are already working on to provide the option and we will update here once we release.
By default we do group dynamic urls in to a single url. For example you have /webpageurl/13t3sjsydd48dh/ and /webpageurl/udjdh838d3d/, we capture these and show it as /webpageurl/*/.
>Recordings, maybe? Snapshots are near useless
Snapshots do show the user navigation path by browser and location. Performance metrics for each page like its response time, resource load time etc. are also shown. However we do accept that recordings will be even more insightful, so we will check the feasibility and number of such feature requests and add it to our roadmap.
>Javascript warnings?
These are also not straightforwardly possible to fetch as it is a browser functionality and it appears in the console. We will look into the feasibility of intercepting the console API of browser and try to fetch these.
>In the section "Performance affected by various factors", we do have all the metrics you have mentioned there.
We do show the CDN metrics filtered by domains in the Resources tab in RUM client. Aggregated CDN requests are one thing we are working on.
We also show error frequency and types along with browsers and device types in javascript errors tab.
All the Performance metrics at various steps in the browser's functionality like the connection time, DNS time, backend time(server time), firstbyte time, download time, document load time, page rendering time are captured for every page request and are available.
We also have RUM traceroute where you can identify the network path a request took to get to your website, and every hop made in between. This helps you to identify whether requests experienced any interruptions reaching your website. For details, you can refer https://www.site24x7.com/help/apm/rum/rum-traceroute.html
Regards
Jaya Simha
APM Insight-RUM
Hi Jaya
First of all thanks for getting back to me so soon after and please excuse me for not getting back to you sooner.
I'm just gonna list the headings and add comments:
> "Share of cacheable and non-cacheable resources" and "Cached vs uncached resources"
I agree with you... However, maybe for response header headers, you could sample requests, e.g. map all the included resources and check them randomly. Alternatively, this could be offloaded to server side checking of headers, so it's not the client testing the server response headers.
Next, for browser cache, maybe this could be used as well?:
stackoverflow.com/questions/260043/how-can-i-use-javascript-to-detect-if-i-am-on-a-cached-page
>Resource download size? How else would I know how performance is? Timing alone is not enough, is it?
Great news. Looking forward to this!
>Improved resources reporting, enabling exploration of requests and headers
Regardless of what you say, your RUM resource reporting, exploration options and so on can definitely be improved. Exploration also extends to actually being able to filter requests - However, I fully understand that you cannot base it on client side reporting as this information is not available in the browser.
>Use filtering/segments such as device type. There's a Device Type report, but it's very basic, Page groups and filtering
First of all, I'm looking forward to seeing your improvements.
I will elaborate on my requirements: Instead of having separate reporting views such as device type, geo, etc. as the turning point for reporting, please consider the scenario where I want to inspect certain resources such as all resources served via cdn.example.com and the performance of the served files in different markets. Being able to filter, explore and analyze in a flexible manner will help me evaluate different vendors, pinpoint slow vendors, bottle necks, resource hogs, etc.
I hope it makes sense... otherwise, please contact me.
>Recordings, maybe? Snapshots are near useless
I think the usefulness of snapshots boils down to some of the issues mentioned above. E.g. It's difficult to use the snapshots to pinpoint what causing a slow. I'll give you an example:
Regardless of which snapshot I explore, I see only 5 domains in the "Domain And Resources Info" even though the page is requesting resources from many more domains.
In order for snapshots to be truly useful, it should be a waterfall view and include all requests in a sortable, filterable manner.
Additionally, the snapshots lists pages viewed. Now, let's say I want to find out if the page is slow because it's being loaded by users in a "slow" country vs having elements which are slowing the page vs a combination of these 2 factors, I can't. If I go into the page report (yes, I have to switch views/reports as well) and try to find the URL using copy/paste - I can't find it (in this case, even though the time period is the same)! This leads me to believe that the two reports are not in sync. Anyway, I can then explore a similar page but I can't see what's going on on that page... the summary gives an high level overview which is nice, ajax calls is the next tab which is empty and if I'm lucky, there might be a few snapshots available on the snapshot page. However, I want to see a detailed breakdown of that page... and yes, I know it'll change over time so depending on the time frame I want to explore, some resources are likely to be different.
How can that be handled? Well, I guess you could create a general overview where content types can be explored, another where it can be explored by domain, an aggregated view of all resources that have been loaded over the time frame (with percentages of how many times each resource was requested) and a view where all variable resources have been filtered out.
I have tons of ideas for improvements. It's really up to you to view the challenge in a new light and start taking your own poison, because I bet you wouldn't find your RUM tool very useful either.
>Javascript warnings?
I understand. Hope you'll find a way because it would be very helpful in detecting deprecation warnings, etc.
>In the section "Performance affected by various factors"
I've seen it but it doesn't help. I can't explore the resources. E.g. "First Party Resources", if I select a domain to explore, I can't see what resources were requested from that domain :'(
In essence, the Site24x7 RUM is reporting on a very superficial level. I do understand and respect that granularity might lead to slower interface and a poor user experience, so I'll leave it up to you to prioritize the level of detail.
All I'm saying is that your RUM has potential to be really really useful, acting as a tool to track down and eliminate requests which are slowing down the apps.
I'm looking forward to seeing my pain points being addressed ;-)
Thanks for listening.
Hi Allan
Thank you for sharing your insights with us. Most of the features we discussed are in our roadmap. We will keep you updated in the thread.
Regards
Jaya Simha
APM Insight-RUM