My First Experience With SigNoz and OpenTelemetry

 

My First Experience With SigNoz and OpenTelemetry

When I first opened the SigNoz website, I was confused.

I saw words like observability, OpenTelemetry, distributed traces, metrics, and logs. They sounded like advanced topics meant only for professional developers working at large companies.

I thought, “Do I really need to understand all of this for a small project?”

After spending some time learning the basics, I realized that SigNoz is not as scary as it first looks.

The main idea is actually simple:

SigNoz helps you see what is happening inside your application.

It can show why an application is slow, where an error happened, how many requests failed, and which part of the system is creating a problem.

In this blog, I will explain SigNoz in simple language, share what I learned about OpenTelemetry, and talk about the feature I found most useful.

Why Applications Need Monitoring

Imagine you build a chatbot.

A user types a question, but the chatbot takes eight seconds to answer.

You know that it is slow, but you do not know why.

There could be many possible reasons:

  • The AI model might be slow.
  • The database search might take too long.
  • An external API might not respond.
  • The application might call the same tool more than once.
  • The server might not have enough memory.
  • An error might happen during one of the steps.

Without a monitoring tool, you may need to check many files and read a lot of code.

You might change one part of the app, test it again, and hope that the problem is fixed.

That is mostly guessing.

SigNoz gives you real information from the running application, so you can look for evidence instead.

For example, instead of saying:

“I think the database is slow.”

You may be able to say:

“The trace shows that the database took 200 milliseconds, but the AI model request took four seconds.”

That is a much clearer answer.

What Is SigNoz?

SigNoz is an open-source observability platform.

The word observability sounds complicated, but it simply means understanding what is happening inside a system by looking at the information it creates.

SigNoz can help developers monitor:

  • Application errors
  • Response time
  • Failed requests
  • Slow operations
  • CPU usage
  • Memory usage
  • Application logs
  • Request traces
  • Service performance

Think about a car.

When you drive a car, the dashboard shows important information such as:

  • Speed
  • Fuel level
  • Engine temperature
  • Warning lights

Without the dashboard, the car may still move, but you would not know when something is wrong.

SigNoz works like a dashboard for software.

Your application continues running, while SigNoz collects and displays information about its health and performance.

It does not build your application.

It does not replace your AI model.

It does not create a website for you.

Its job is to help you monitor, understand, and debug the application you already built.

What Is OpenTelemetry?

SigNoz often works together with OpenTelemetry.

OpenTelemetry is a system used to collect information from an application.

This information can include:

  • Logs
  • Metrics
  • Traces

OpenTelemetry then sends this data to a monitoring platform such as SigNoz.

The easiest way to remember the difference is:

OpenTelemetry collects the information. SigNoz helps you view and understand it.

You can also think of OpenTelemetry as a delivery service.

Your application creates useful information. OpenTelemetry picks it up and sends it to SigNoz. SigNoz then organizes the data into dashboards, charts, logs, and traces.

This is helpful because developers can use a common standard instead of adding a completely different monitoring system for every project.

Logs, Metrics, and Traces

These are three important parts of observability.

At first, they sounded very similar to me. But each one answers a different question.

Logs: What Happened?

Logs are messages created by an application while it is running.

For example:

User opened the chatbot
AI request started
Document uploaded successfully
Database connection failed
API request timed out
Response returned to the user

Logs are like the diary of your application.

They record events that happened at a certain time.

Logs are useful when you want to know:

  • Which error message appeared
  • When an event happened
  • Which user action started the problem
  • Whether an API request failed
  • Whether a task completed successfully

For example, imagine a chatbot stops working.

A log may show:

Error: Invalid API key

Now you know that the problem is not the chatbot design. The application cannot connect to the AI service because the key is missing or incorrect.

Logs answer the question:

What happened?

Metrics: How Is the Application Performing?

Metrics are numbers measured over time.

Examples of metrics include:

  • Total number of requests
  • Average response time
  • Number of failed requests
  • CPU usage
  • Memory usage
  • Error percentage
  • Requests per minute
  • Number of active users

Metrics help you understand the bigger picture.

For example, one slow request may not be a serious problem.

But imagine the average response time changes from one second to five seconds. That may show that the whole application is becoming slower.

A metric could also show that the error rate increased after a new update.

Metrics answer questions such as:

  • Is the app becoming slower?
  • Are errors increasing?
  • Is traffic growing?
  • Is the server using too much memory?
  • How many users are sending requests?

Metrics answer the question:

How is the application performing over time?

Traces: Where Did the Request Go?

A trace shows the complete journey of one request.

Imagine a user asks an AI chatbot:

“Explain photosynthesis.”

The application may need to complete several steps:

  1. Receive the question.
  2. Check the user input.
  3. Search stored documents.
  4. Call an AI model.
  5. Format the response.
  6. Send the answer back.

A trace can show how long each step took.

For example:

Receive question: 10 ms
Check input: 15 ms
Search documents: 250 ms
Call AI model: 3.8 seconds
Format response: 40 ms
Return answer: 20 ms

Now it is easy to see that the AI model call used most of the time.

The other parts of the app were fast.

Without a trace, you would only know that the entire request took around four seconds.

With a trace, you can see exactly where those four seconds were spent.

Traces answer the question:

Where did the request go, and which step was slow or broken?

How Logs, Metrics, and Traces Work Together

Logs, metrics, and traces are useful on their own, but they become more powerful when used together.

Imagine that your dashboard shows a sudden increase in failed requests.

You could investigate the problem like this:

  1. Metrics show that the error rate increased.
  2. Traces show which operation failed.
  3. Logs show the exact error message.

For example:

  • The metric says 15% of requests failed.
  • The trace says the AI model call failed.
  • The log says the request failed because of a timeout.

Now you understand the full problem.

You are no longer searching through random code.

You are following a clear path from the warning to the cause.

Why SigNoz Can Be Useful for AI Applications

AI applications often have many hidden steps.

A normal user only sees the final answer, but the application may be doing many things in the background.

It may:

  • Call an AI model
  • Search a database
  • Read a document
  • Use an external API
  • Call a tool
  • Store information
  • Create another AI request
  • Format the final response

If one step becomes slow, the whole application may feel slow.

If one tool fails, the whole workflow may stop.

This is why observability is important for AI applications.

SigNoz can help developers check:

  • How long an AI request takes
  • Which model call is slow
  • Whether a tool call failed
  • How often users receive errors
  • Which part of the workflow uses the most time
  • Whether the system becomes slower during high traffic
  • Whether the same task is being repeated

This is especially useful when building AI agents.

An AI agent may make several decisions and tool calls before returning an answer. Without traces, it can be difficult to understand what the agent did.

My Favourite SigNoz Feature: Tracing

The SigNoz feature I find most interesting is tracing.

The reason is simple: tracing turns a complicated request into a step-by-step story.

Imagine a chatbot takes six seconds to answer.

Without tracing, you only know this:

“The chatbot is slow.”

That information does not help very much.

With tracing, you might see:

Receive user message: 20 ms
Search database: 300 ms
Call weather API: 500 ms
Call AI model: 4.8 seconds
Create response: 100 ms

Now you know that the AI model call caused most of the delay.

This helps you decide what to improve.

You could try:

  • Using a faster model
  • Making the prompt shorter
  • Removing an unnecessary model call
  • Caching repeated results
  • Running independent tasks at the same time
  • Reducing the amount of data sent to the model

Tracing does not automatically fix the problem.

But it shows you where to start.

That is much better than changing random parts of the code.

Tracing turns “my app is slow” into “this exact step is slow.”

What a Span Means

Inside a trace, each smaller operation is called a span.

A span might represent:

  • A database query
  • An API request
  • An AI model call
  • A tool call
  • A document search
  • A function inside the application

Each span can include useful details, such as:

  • Operation name
  • Start time
  • End time
  • Duration
  • Success or failure status
  • Error information
  • Service name

A full trace is made of several connected spans.

For example:

Chat request
├── Check user input
├── Search documents
├── Call AI model
└── Return response

This makes it easier to understand how the application handled one user request.

Dashboards in SigNoz

Dashboards help developers view important information in one place.

A simple dashboard for an AI chatbot could include:

  • Total requests
  • Successful requests
  • Failed requests
  • Average response time
  • Slowest operation
  • AI model response time
  • CPU usage
  • Memory usage

A dashboard is useful because developers do not need to search for every number manually.

They can open one page and quickly understand how the application is performing.

However, more charts do not always mean a better dashboard.

A dashboard with five useful charts is better than a dashboard with twenty charts that nobody understands.

Each chart should answer a real question.

For example:

  • Are errors increasing?
  • Is the app slower today?
  • Which endpoint receives the most traffic?
  • Which model call takes the most time?

Add your screenshot here: A SigNoz dashboard showing your application data.

Suggested image alt text:

SigNoz dashboard showing AI application performance metrics

Alerts in SigNoz

Developers cannot watch a dashboard every minute.

That is why alerts are useful.

An alert tells you when something important happens.

For example:

  • The error rate becomes higher than 5%.
  • The application response time becomes longer than three seconds.
  • CPU usage becomes too high.
  • A service stops sending data.
  • Too many requests fail in a short period.

An alert can help developers notice a problem before many users complain.

However, the alert limit should be chosen carefully.

If the limit is too low, you may receive warnings for normal behaviour.

If the limit is too high, you may not receive a warning until the problem becomes serious.

A good alert should be connected to something that matters to users.

For example, a response-time alert is useful because users may leave an application when it becomes too slow.

What I Learned From SigNoz

Before learning about SigNoz, I thought monitoring only meant checking whether a server was online.

Now I understand that monitoring and observability can tell us much more.

They can help answer questions such as:

  • Why did this request fail?
  • Which part of the app is slow?
  • When did the problem begin?
  • Is the error happening to one user or many users?
  • Did a new update make the app slower?
  • Which service needs attention?

The biggest lesson for me was that debugging becomes easier when you have real data.

Logs show what happened.

Metrics show the overall pattern.

Traces show the full journey of one request.

Together, they give developers a much clearer picture of the application.

Is SigNoz Only for Large Companies?

No.

Large companies may use observability tools for complex systems, but beginners can also learn from them.

You do not need to start with ten services and a huge AI agent.

You can begin with:

  1. One small application
  2. One API endpoint
  3. One successful request
  4. One failed request
  5. One trace
  6. One dashboard
  7. One simple alert

This is enough to understand the main ideas.

Once you understand how one request appears in SigNoz, you can slowly add more features.

Challenges Beginners May Face

SigNoz can still feel confusing at first.

You may face problems such as:

  • Docker containers not starting
  • Not enough computer memory
  • No traces appearing in the dashboard
  • An incorrect OpenTelemetry endpoint
  • A wrong service name
  • Missing environment variables
  • Logs that do not include enough information

These problems are normal when learning a new tool.

The important thing is to solve one problem at a time.

For example, when no trace appears, you can check:

  • Is SigNoz running?
  • Is the application running?
  • Is OpenTelemetry installed?
  • Is the endpoint correct?
  • Did you send a request to the application?
  • Is the service name visible in SigNoz?

This simple checklist can save a lot of time.

My Main Takeaway

SigNoz may look complicated when you first open it.

There are graphs, traces, filters, dashboards, services, and settings.

But you do not need to understand everything immediately.

Start by learning the three main signals:

  • Logs
  • Metrics
  • Traces

Then open one trace and study the steps.

Look at one metric and notice how it changes.

Read one log and understand what happened.

That is a good beginning.

My main takeaway is:

Observability helps developers replace guessing with evidence.

When an application is slow or broken, SigNoz helps show where the problem may be.

Final Thoughts

AI applications can hide a lot of work behind one simple button.

A user may only see a question box and an answer, but the application may use models, databases, APIs, tools, and documents in the background.

When something goes wrong, developers need a way to understand those hidden steps.

OpenTelemetry collects useful information from the application.

SigNoz turns that information into logs, metrics, traces, dashboards, and alerts.

The feature I find most useful is tracing because it clearly shows the journey of one request.

It helps answer an important question:

Where did the time go?

For a beginner, SigNoz may take some time to understand. But once the basic ideas become clear, it feels much more practical.

You do not need to become an observability expert on the first day.

Start with a small application, send a few requests, and explore the results.

That is how learning begins.

Comments

Post a Comment