Skip to main content

KPIs and metrics

Prerequisite

We highly suggest to learn how to create and run monitorings in our Monitoring basics section before checking the different metrics generated by our system.
 

Overview

When a test is selected and run as a monitoring, kapptivate automatically generates key metrics based on:
  •  Test content
  •  Monitoring result status
  •  Test duration
  •  Resource selected
  •  Worspace and run environment
These metrics can be visualised in a variety of manner under Analytics Custom dashboard​
Analytics custom dashboard
 

Global Test metrics

Common Tags & Fields

The following tags and fields are available on all the global test metrics
Common Tags   Common Fields  
product_id: ID of the product duration_seconds: Duration in seconds
product_name: Name of the product duration_seconds: Duration in seconds
platform_id: ID of the platform duration_seconds: Duration in seconds
platform_name: Name of the platform duration_seconds: Duration in seconds
test_id: ID of the test played duration_seconds: Duration in seconds
test_name: Name of the test played duration_seconds: Duration in seconds
monitoring_id: Id of monitoring duration_seconds: Duration in seconds
monitoring_name: Name of monitoring duration_seconds: Duration in seconds
subtest_id: ID of the test directly containing the action, or empty if it's the top level test duration_seconds: Duration in seconds
subtest_name: name of the test directly containing the action, or empty if it's the top level test duration_seconds: Duration in seconds
type: test suite / monitoring duration_seconds: Duration in seconds
zone: Location of the action duration_seconds: Duration in seconds
status: Status of the test or action ("succes", "failed", "error") duration_seconds: Duration in seconds
failure_reason: Failure reason duration_seconds: Duration in seconds

Metrics

Metric Tags   Fields
test (created for each test, and also each subtest)
failed_action_kind:
Failed action kind ("USSD", "SMS"…)
  
 
_status*:
"succes","failed", "error"
  
action (created for each action, including subtest call action)
action_index:
 Index of the action  
 
action_kind:
 Kind of the action  
 
action_owner:
Owner of the action (IMSI for SIM or ID for the rest)
  
 
_status:
  •  cellular_ussd_status
  •  cellular_ussd_push_status
  •  cellular_sms_reception_status
  •  cellular_sms_sending_status
  •  cellular_sms_delivery_report_status
  •  cellular_call_wait_status
  •  cellular_call_place_status
  •  cellular_pdp_activation_status
  •  cellular_dns_status
  •  cellular_http_download_status
  •  cellular_http_upload_status
  •  cellular_ftp_download_status
  •  cellular_ftp_upload_status
  •  cellular_email_send_status
  •  cellular_ping_status
  •  cellular_tcp_ping_status
  •  cellular_http_ping_status
  •  cellular_http_api_status
  •  cellular_speed_test_status
  •  cellular_traceroute_status
  •  ethernet_dns_status
  •  ethernet_email_sending_status
  •  ethernet_ftp_download_status
  •  ethernet_ftp_upload_status
  •  ethernet_http_api_status
  •  ethernet_http_download_status
  •  ethernet_http_upload_status
  •  ethernet_http_ping_status
  •  ethernet_ping_status
  •  ethernet_tcp_ping_status
  •  ethernet_speed_test_status
  •  ethernet_traceroute_status
  •  ethernet_web_script_status
  •  smartphone_web_mobile_script_status
  •  smartphone_app_script_status
  •  smartphone_ios_app_script_status
  •  ownerless_test_call
 

Cellular test KPI

Common Tags & Fields

The following tags and fields are available on all the cellular test metrics
Common Tags   Common Fields  
product_id: ID of the product duration_seconds: Duration in seconds
product_name: Name of the product duration_seconds: Duration in seconds
platform_id: ID of the platform duration_seconds: Duration in seconds
platform_name: Name of the platform duration_seconds: Duration in seconds
test_id: ID of the test played duration_seconds: Duration in seconds
test_name: Name of the test played duration_seconds: Duration in seconds
action_index: Index of the action duration_seconds: Duration in seconds
action_owner: Owner of the action (IMSI for SIM or ID for the rest) duration_seconds: Duration in seconds
zone: Location of the action duration_seconds: Duration in seconds
access_technology: Kind of access technology duration_seconds: Duration in seconds
status: Status of the test or action ("succes", "failed", "error") duration_seconds: Duration in seconds
failure_reason: Failure reason duration_seconds: Duration in seconds
subtest_id: ID of the test directly containing the action, or empty if it's the top level test duration_seconds: Duration in seconds
subtest_name: name of the test directly containing the action, or empty if it's the top level test duration_seconds: Duration in seconds
apn: apn used duration_seconds: Duration in seconds
dns_server: dns server used duration_seconds: Duration in seconds
 

Metrics

Metric Tags   Fields  
cellular_ussd        
cellular_ussd_push       ​
cellular_sms_sending        
cellular_sms_delivery_report       ​
cellular_sms_reception        
cellular_pdp_activation       ​
cellular_dns host: Host to be resolved namelookup_duration: The time, in seconds, it took from the start until the name resolving was completed
cellular_ping host: Host to be pinged packet_loss_percent Packet loss percentage
  packet_size: Size of the packet sent rtt_seconds RTT mean
      jitter Jitter
cellular_tcp_ping host: Host to be pinged packet_loss_percent Packet loss percentage
      rtt_seconds RTT mean
cellular_http_ping host: Host to be pinged packet_loss_percent Packet loss percentage
      rtt_seconds RTT mean
cellular_http_api url: URL namelookup_duration: The time, in seconds, it took from the start until the name resolving was completed
  http_method: HTTP Method tcpconnection_duration: The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed.
  status_code: HTTP Status code contentgeneration_duration: The time, in seconds, server side generation content
      contenttransfer_duration: The time, in seconds, it took from the beginning until the end of the transfer.
      http_call_duration: Total time in seconds of request completed.
      connect_duration: The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed.
      appconnect_duration: The time, in seconds, it took from the start until the SSL/SSH/etc connect/handshake to the remote host was completed
      pretransfer_duration: The time, in seconds, it took from the start until the file transfer was just about to begin. This includes all pre-transfer commands and negotiations that are specific to the particular protocol(s) involved.
      redirect_duration: The time, in seconds, it took for all redirection steps include name lookup, connect, pretransfer and transfer before the final transaction was started. time_redirect shows the complete execution time for multiple redirections.
      starttransfer_duration: The time, in seconds, it took from the start until the first byte was just about to be transferred. This includes time_pretransfer and also the time the server needed to calculate the result.
cellular_http_download url: URL namelookup_duration: The time, in seconds, it took from the start until the name resolving was completed
  status_code: HTTP Status code tcpconnection_duration: The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed.
      contentgeneration_duration: The time, in seconds, server side generation content
      contenttransfer_duration: The time, in seconds, it took from the beginning until the end of the transfer.
      http_call_duration: Total time in seconds of request completed.
      connect_duration: The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed.
      appconnect_duration: The time, in seconds, it took from the start until the SSL/SSH/etc connect/handshake to the remote host was completed
      pretransfer_duration: The time, in seconds, it took from the start until the file transfer was just about to begin. This includes all pre-transfer commands and negotiations that are specific to the particular protocol(s) involved.
      redirect_duration: The time, in seconds, it tookfor all redirection steps include name lookup, connect, pre transfer and transfer before the final transaction was started. time_redirect shows the complete execution time for multiple redirections.
      starttransfer_duration: The time, in seconds, it took from the start until the first byte was just about to be transferred. This includes time_pretransfer and also the time the server needed to calculate the result.
      througput_kbps: Throughput in kbps
cellular_http_upload url: URL througput_kbps: Throughput in kbps
cellular_ftp_download url: URL througput_kbps: Throughput in kbps
cellular_ftp_upload url: URL througput_kbps: Throughput in kbps
cellular_email_sending url: URL    
cellular_call_wait caller_number: The phone number of the caller cellular_call_duration: The duration of the call that was received, in seconds.
  call_receiver_number: The phone number of the call receiver cellular_call_connect_duration: The duration,in seconds, that it took for the incoming call to reach the SIM.
cellular_call_place caller_number: The phone number of the caller cellular_call_duration: The duration of the call that was received, in seconds.
  call_receiver_number: The phone number of the call receiver cellular_call_connect_duration: The duration,in seconds, that it took for the incoming call to reach the SIM.
      cellular_call_post_dial_delay: The duration, in seconds, that it took between the moment the call was started by the caller and the moment the callee received the call's first ring.
cellular_streaming_video url: URL total_bytes: Total bytes of streaming video
      read_bytes: Total read bytes from streaming video
      displayed_pictures: Pictures displayed from streaming
      lost_pictures: Pictures lost from streaming
      played_audio_buffers: Audio played from streaming
      lost_audio_buffers: Lost audio from streaming
      first_buffering_time: Time in seconds, which represent the time of first media load
      additional_buffering_time: Time in seconds, which represent the time of additional media load
      additional_buffering_count: Number of additional buffering
      played_time: Time in seconds, which represent the time of played video
      lost_pictures_percent: Percentage of lost pictures
      buffering_percent: Percentage of buffering 
​| Tables   |      Are      |  Cool |
|----------|:-------------:|------:|
| col 1 is |  left-aligned | $1600 |
| col 2 is |    centered   |   $12 |
| col 3 is | right-aligned |    $1 |

Ethernet test KPI

Common Tags & Fields

The following tags and fields are available on all the ethernet test metrics
 
Common Tags   Common Fields  
product_id: ID of the product duration_seconds: Duration in seconds
product_name: Name of the product duration_seconds: Duration in seconds
platform_id: ID of the platform duration_seconds: Duration in seconds
platform_name: Name of the platform duration_seconds: Duration in seconds
test_id: ID of the test played duration_seconds: Duration in seconds
test_name: Name of the test played duration_seconds: Duration in seconds
action_index: Index of the action duration_seconds: Duration in seconds
action_owner: Owner of the action (IMSI for SIM or ID for the rest) duration_seconds: Duration in seconds
zone: Location of the action duration_seconds: Duration in seconds
status: Status of the test or action ("succes", "failed", "error") duration_seconds: Duration in seconds
failure_reason: Failure reason duration_seconds: Duration in seconds
subtest_id: ID of the test directly containing the action, or empty if it's the top level test duration_seconds: Duration in seconds
subtest_name: name of the test directly containing the action, or empty if it's the top level test duration_seconds: Duration in seconds
dns_server: dns server used duration_seconds: Duration in seconds
 

Metrics

Metric Tags   Fields  
ethernet_dns host: Host to be resolved namelookup_duration: The time, in seconds, it took from the start until the name resolving was completed
ethernet_ping host: Host to be pinged packet_loss_percent Packet loss percentage
  packet_size: Size of the packet sent rtt_seconds RTT mean
      jitter Jitter
ethernet_tcp_ping host: Host to be pinged packet_loss_percent Packet loss percentage
      rtt_seconds RTT mean
ethernet_http_ping host: Host to be pinged packet_loss_percent Packet loss percentage
      rtt_seconds RTT mean
ethernet_http_api url: URL namelookup_duration: The time, in seconds, it took from the start until the name resolving was completed
  http_method: HTTP Method tcpconnection_duration: The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed.
  status_code: HTTP Status code contentgeneration_duration: The time, in seconds, server side generation content
      contenttransfer_duration: The time, in seconds, it took from the beginning until the end of the transfer.
      http_call_duration: Total time in seconds of request completed.
      connect_duration: The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed.
      appconnect_duration: The time, in seconds, it took from the start until the SSL/SSH/etc connect/handshake to the remote host was completed
      pretransfer_duration: The time, in seconds, it took from the start until the file transfer was just about to begin. This includes all pre-transfer commands and negotiations that are specific to the particular protocol(s) involved.
      redirect_duration: The time, in seconds, it took for all redirection steps include name lookup, connect, pretransfer and transfer before the final transaction was started. time_redirect shows the complete execution time for multiple redirections.
      starttransfer_duration: The time, in seconds, it took from the start until the first byte was just about to be transferred. This includes time_pretransfer and also the time the server needed to calculate the result.
ethernet_http_download url: URL namelookup_duration: The time, in seconds, it took from the start until the name resolving was completed
  status_code: HTTP Status code tcpconnection_duration: The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed.
      contentgeneration_duration: The time, in seconds, server side generation content
      contenttransfer_duration: The time, in seconds, it took from the beginning until the end of the transfer.
      http_call_duration: Total time in seconds of request completed.
      connect_duration: The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed.
      appconnect_duration: The time, in seconds, it took from the start until the SSL/SSH/etc connect/handshake to the remote host was completed
      pretransfer_duration: The time, in seconds, it took from the start until the file transfer was just about to begin. This includes all pre-transfer commands and negotiations that are specific to the particular protocol(s) involved.
      redirect_duration: The time, in seconds, it took for all redirection steps include name lookup, connect, pretransfer and transfer before the final transaction was started. time_redirect shows the complete execution time for multiple redirections.
      starttransfer_duration: The time, in seconds, it took from the start until the first byte was just about to be transferred. This includes time_pretransfer and also the time the server needed to calculate the result.
      througput_kbps: Throughput in kbps
ethernet_http_upload url: URL througput_kbps: Throughput in kbps
ethernet_ftp_download url: URL througput_kbps: Throughput in kbps
ethernet_ftp_upload url: URL througput_kbps: Throughput in kbps
ethernet_email_sending url: URL    
ethernet_web_script        
ethernet_web_script_step_performance   The name of the step in the test script    
  step_status The status of the step in the test script. ("Success" or "failed")    
ethernet_streaming_video url: URL total_bytes: Total bytes of streaming video
      read_bytes: Total read bytes from streaming video
      displayed_pictures: Pictures displayed from streaming
      lost_pictures: Pictures lost from streaming
      played_audio_buffers: Audio played from streaming
      lost_audio_buffers: Lost audio from streaming
      first_buffering_time: Time in seconds, which represent the time of first media load
      additional_buffering_time: Time in seconds, which represent the time of additional media load
      additional_buffering_count: Number of additional buffering
      played_time: Time in seconds, which represent the time of played video
      lost_pictures_percent: Percentage of lost pictures
      buffering_percent: Percentage of buffering

Smartphone application test KPI

Common Tags & Fields

The following tags and fields are available on all the smartphone application test metrics
Common Tags   Common Fields  
product_id: ID of the product duration_seconds: Duration in seconds
product_name: Name of the product duration_seconds: Duration in seconds
platform_id: ID of the platform duration_seconds: Duration in seconds
platform_name: Name of the platform duration_seconds: Duration in seconds
test_id: ID of the test played duration_seconds: Duration in seconds
test_name: Name of the test played duration_seconds: Duration in seconds
action_index: Index of the action duration_seconds: Duration in seconds
action_owner: Owner of the action (IMSI for SIM or ID for the rest) duration_seconds: Duration in seconds
zone: Location of the action duration_seconds: Duration in seconds
status: Status of the test or action ("succes", "failed", "error") duration_seconds: Duration in seconds
failure_reason: Failure reason duration_seconds: Duration in seconds
subtest_id: ID of the test directly containing the action, or empty if it's the top level test duration_seconds: Duration in seconds
subtest_name: name of the test directly containing the action, or empty if it's the top level test duration_seconds: Duration in seconds

Metrics

Metric Tags   Fields
smartphone_app_script      
smartphone_script_step_performance
step_name:
The name of the step in the android smartphone script
  
 
step_status:
The status of the step in the android smartphone script ("success" or "failed")
  
smartphone_ios_web_script_step_performance
step_name:
The name of the step in the ios smartphone script and web script
  
 
step_status:
The status of the step in the ios smartphone script and web script ("success" or "failed")
  
 
Was this article helpful?
Powered by Zendesk