Dynatrace is one of the leading Application Performance Monitors (APM) on the market, capable of intercepting all the traffic of your applications, to detect performance problems and indicate specifically when and why an error occurred, even at the level of code and database queries.
In this article, we show you how to improve incident response by getting your Dynatrace alarms notified by phone calls using 24Cevent software.
Improve your incident response times
One of Dynatrace’s key features is its ability to generate real-time alarms when system problems are detected. These alarms are sent through various channels, such as email, SMS or webhooks, however, phone calls are not natively available. Not having an automatic telephone call system means that any critical incident detected during non-business hours will not be attended to until the beginning of the next working day.
If we want to improve platform uptime and customer experience, improving response times and reducing MTTR (mean time to repair) is one of the most important indicators to achieve this.
Automatic notifications by phone call play 2 critical roles in this process:
- Reduce MTTA (mean time to acknowledge), where by having a predefined decision matrix, the time from when an incident is detected until the person in charge is notified is reduced to a couple of seconds.
- Reduce MTTR by being able to have 24×7 support shifts in case of events.
Set up the integration in Dynatrace
As a prerequisite, you must have an account created in 24Cevent, the platform that will automatically notify Dynatrace alarms by phone calls 24×7. There must also be TCP communication between the Dynatrace source and the 24Cevent destination.
To configure Dynatrace, once you have logged in, go to Settings –> Integrations –> Problem Notifications.
Then select the “Add notification” box.
Once in the new notification rule creation menu, select “Notification Type”, “Custom integration”. Then name the new rule in the “Display name” field.
NOTE: To name a notification rule in Dynatrace, you should use the following format (recommended): Destination Tool – Name of the Alerting Profile – 24c Service. For example, 24c – Default – dynatrace_poc_service
In the “Webhook URL” field, you must assign the following endpoint created in the integration. From the buttons below, only enable the one for “call webhook if problem is closed”
https://app.24cevent.com/proxy/forwarder/json/?access_token=mbHwdWnlbQh5Xjxu09poeFSgTbHTLs&channel=57abe038374620c2875ebf62ae2027cca5463f25c722be4435fa7eb7c8f456d0&no_receiver
Once you get to the part of adding the Custom Payload, you must fill in the following information:
{
"State":"{State}",
"PID":"{PID}",
"ProblemTitle":"{ProblemTitle}",
"ProblemSeverity":"{ProblemSeverity}",
"ProblemImpact":"{ProblemImpact}",
"ImpactedEntity":"{ImpactedEntity}",
"ImpactedEntityNames":"{ImpactedEntityNames}",
"ProblemURL":"{ProblemURL}",
"ProblemDetailsText":"{ProblemDetailsText}",
"ProblemDetailsJSONv2":{ProblemDetailsJSONv2},
"Resolutor":"dynatrace_poc_service",
"ManagementZone":"all",
"AlertType":"{State}{ProblemSeverity}",
"Message":"Dynatrace notifica la alerta: {ImpactedEntity}",
"ExternalId":"{PID}",
"CustomSubject":"Alerta Urgente",
"Tags":"{Tags}",
"NamesOfImpactedEntities":"{NamesOfImpactedEntities}"
}NOTA: Where we have to manually fill in the following fields:
- “Resolutor:” Name of the 24c service where the alert will arrive.
- “ManagementZone:” Management Zone to which this alert belongs (optional, can be left blank “”).
- “CustomSubject:” This element is used to add a custom subject to the email that will be sent (optional).
Finally, in the Alerting Profile dropdown list, select the alert profile you want, or you can leave the default one which will notify for all problems. Save the changes and that’s it.
Once the integration is ready, go back to 24Cevent to see how your alerts are being received.
Within 24Cevent
In the “Operations” section, you will be able to see how all your alerts from Dynatrace are coming in.
To set up a phone call, you must follow these steps:
1) Create a contact.
In the “Contacts” section — fill in the fields: Name, phone number, and email — Create contact.
2) Crear un turno (horario de trabajo de este contacto).
In the “Work Teams” section — New shift.
Fill in the required fields: Shift name — Select created contact — And schedule — Click the “Save” button.
3) Create a notification flow.
Go to “Notification flows” — New notification flow.
Fill in the required fields: Flow name, Responsible shift, and Drag the “Notification rule”, which in this case is the call. Finally, click save.
On the main panel, you will be able to see your active notification flow:
And that’s it! You will start receiving phone calls for your Dynatrace alerts:
Demo
In this video, we do a live demonstration of how the Dynatrace-24Cevent integration works to generate alarm notifications via phone calls.
If you cannot view the video, you can watch it here: https://youtu.be/BMbzoQ6jA_8?si=MMjdWNWHtRug98V8
Conclusion
Dynatrace alarm notification by phone calls through the 24Cevent integration tool is an effective strategy for improving incident response in IT environments. It provides a quick and direct way to alert operations teams to critical issues, resulting in faster resolution and less business disruption. By leveraging this functionality, organizations can strengthen their observability and monitoring capabilities, ensuring the stability and performance of their IT systems.
Early detection of an incident and knowing the root cause is critical. But just as important is that the alarm is handled in a timely manner and that the person trained to resolve the incident is notified as soon as possible to begin working on a solution.
Therefore, implementing an automated phone call notification solution as a complement to native notification methods can provide several benefits:
- 24×7: To be able to provide 24×7 support to the systems at a low cost.
- MTTA: Reducing notification times
- Standardization: To be able to deliver standardized context information to the support team.
- Parallelization: Notifying different equipment simultaneously, including suppliers
- SLA Control: Measure the time elapsed between the generation of an incident and its resolution, in order to generate SLA compliance reports.
Do you need to ensure that your Dynatrace alarms will be managed in a timely manner? Forget about hiring profiles to watch screens day and night, integrate your alerts to 24Cevent, classify them and generate real-time notifications to your support team when incidents occur.
Join today, at no cost and with no commitments in our free level and discover how with 24Cevent you can have your own 24×7 automated operations center.
