Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Crystal::Tracing for runtime tracing #14659

Merged
merged 19 commits into from
Jun 15, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
b55d66c
Add Crystal::System::Thread.current_thread? + Thread#scheduler?
ysbaddaden May 27, 2024
824e2af
Initial Crystal.trace implementation
ysbaddaden May 27, 2024
eddc005
Line protocol with abstract metadata (named tuple)
ysbaddaden May 27, 2024
22f5aac
Trace: print to file (CRYSTAL_TRACE_FILE)
ysbaddaden May 27, 2024
d28e6ce
Fix: Crystal::Tracing.init shall return nil
ysbaddaden Jun 3, 2024
149f3d4
Prefer System.printf_error over LibC.dprintf
ysbaddaden Jun 4, 2024
c8fd378
Fix: no need to prefix the Crystal namespace
ysbaddaden Jun 4, 2024
cbb904f
Fix: remove unused @@startup_tick
ysbaddaden Jun 4, 2024
379b4a5
fixup! Trace: print to file (CRYSTAL_TRACE_FILE)
ysbaddaden Jun 4, 2024
a54c848
Merge branch 'master' into feature/tracing
straight-shoota Jun 6, 2024
5541f6e
Fix: leverage String.each_utf16_char instead of external C call
ysbaddaden Jun 13, 2024
11faaa3
Apply code review suggestion
ysbaddaden Jun 13, 2024
d214022
Replace macro with a couple methods
ysbaddaden Jun 13, 2024
9a9025c
fixup! Fix: leverage String.each_utf16_char instead of external C call
ysbaddaden Jun 13, 2024
9ce8adb
fixup! Replace macro with a couple methods
ysbaddaden Jun 13, 2024
34ee68d
Fix: don't overload Enum#to_s (use distinct method)
ysbaddaden Jun 13, 2024
e8648ef
Fix: use Pointer::Appender in print_error(Slice(UInt8))
ysbaddaden Jun 13, 2024
5b65aad
Fix: use ensure instead of tap so we still print the trace
ysbaddaden Jun 13, 2024
1dfe3c5
Refactor: Section.from_id(slice) and Section#to_id
ysbaddaden Jun 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/concurrent.cr
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ end
# ```
def spawn(*, name : String? = nil, same_thread = false, &block)
fiber = Fiber.new(name, &block)
Crystal.trace :sched, :spawn, fiber: fiber
Crystal.trace :sched, "spawn", fiber: fiber
{% if flag?(:preview_mt) %} fiber.set_current_thread if same_thread {% end %}
fiber.enqueue
fiber
Expand Down
10 changes: 5 additions & 5 deletions src/crystal/scheduler.cr
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ class Crystal::Scheduler
end

def self.enqueue(fiber : Fiber) : Nil
Crystal.trace :sched, :enqueue, fiber: fiber do
Crystal.trace :sched, "enqueue", fiber: fiber do
thread = Thread.current
scheduler = thread.scheduler

Expand All @@ -51,7 +51,7 @@ class Crystal::Scheduler
end

def self.reschedule : Nil
Crystal.trace :sched, :reschedule
Crystal.trace :sched, "reschedule"
Thread.current.scheduler.reschedule
end

Expand All @@ -61,12 +61,12 @@ class Crystal::Scheduler
end

def self.sleep(time : Time::Span) : Nil
Crystal.trace :sched, :sleep, for: time.total_nanoseconds.to_i64!
Crystal.trace :sched, "sleep", for: time.total_nanoseconds.to_i64!
Thread.current.scheduler.sleep(time)
end

def self.yield : Nil
Crystal.trace :sched, :yield
Crystal.trace :sched, "yield"

# TODO: Fiber switching and libevent for wasm32
{% unless flag?(:wasm32) %}
Expand Down Expand Up @@ -115,7 +115,7 @@ class Crystal::Scheduler
end

protected def resume(fiber : Fiber) : Nil
Crystal.trace :sched, :resume, fiber: fiber
Crystal.trace :sched, "resume", fiber: fiber
validate_resumable(fiber)

{% if flag?(:preview_mt) %}
Expand Down
80 changes: 52 additions & 28 deletions src/crystal/tracing.cr
Original file line number Diff line number Diff line change
@@ -1,4 +1,29 @@
module Crystal
# :nodoc:
module Tracing
@[Flags]
enum Section
Gc
ysbaddaden marked this conversation as resolved.
Show resolved Hide resolved
Sched

# Override Enum#to_s to return static strings. Doesn't consider the enum
# has having the flags annotation, since we only need to translate
# `Section::Gc` as `"gc"` in `Crystal.trace`.
def to_s : String
ysbaddaden marked this conversation as resolved.
Show resolved Hide resolved
{% begin %}
case self
{% for constant in @type.constants %}
when {{constant}}
{{constant.id.underscore.stringify}}
{% end %}
else
"???"
end
{% end %}
end
end
end

{% if flag?(:tracing) %}
# :nodoc:
module Tracing
Expand Down Expand Up @@ -51,12 +76,6 @@ module Crystal
end
end

@[Flags]
enum Section
Gc
Sched
end

@@sections = Section::None
@@handle = uninitialized System::FileDescriptor::Handle

Expand Down Expand Up @@ -155,6 +174,14 @@ module Crystal
end

# :nodoc:
#
# Formats and prints a log message to stderr. The generated message is
# limited to 512 bytes (PIPE_BUF) after which it will be truncated. Being
# below PIPE_BUF the message shall be written atomically to stderr,
# avoiding interleaved or smashed traces from multiple threads.
#
# Windows may not have the same guarantees but the buffering should limit
# these from happening.
def self.log(section : String, operation : String, time : UInt64, **metadata) : Nil
buf = BufferIO(512).new
buf.write section
Expand Down Expand Up @@ -199,27 +226,21 @@ module Crystal
end
end

# Formats and prints a log message to stderr. The generated message is
# limited to 512 bytes (PIPE_BUF) after which it will be truncated. Being
# below PIPE_BUF the message shall be written atomically to stderr,
# avoiding interleaved or smashed traces from multiple threads.
#
# Windows may not have the same guarantees but the buffering should limit
# these from happening.
macro trace(section, operation, tick = nil, **metadata, &block)
if ::Crystal::Tracing.enabled?(\{{section}})
%tick = \{{tick}} || ::Crystal::System::Time.ticks
\{% if block %}
%ret = \{{yield}}
%duration = ::Crystal::System::Time.ticks - %tick
::Crystal::Tracing.log(\{{section.id.stringify}}, \{{operation.id.stringify}}, %tick, duration: %duration, \{{metadata.double_splat}})
%ret
\{% else %}
::Crystal::Tracing.log(\{{section.id.stringify}}, \{{operation.id.stringify}}, %tick, \{{metadata.double_splat}})
nil
\{% end %}
def self.trace(section : Tracing::Section, operation : String, time : UInt64? = nil, **metadata, &)
if Tracing.enabled?(section)
time ||= System::Time.ticks
yield.tap do
ysbaddaden marked this conversation as resolved.
Show resolved Hide resolved
duration = System::Time.ticks - time
Tracing.log(section.to_s, operation, time, **metadata, duration: duration)
end
else
\{{yield}}
yield
end
end

def self.trace(section : Tracing::Section, operation : String, time : UInt64? = nil, **metadata) : Nil
if Tracing.enabled?(section)
Tracing.log(section.to_s, operation, time || System::Time.ticks, **metadata)
end
end
{% else %}
Expand All @@ -236,8 +257,11 @@ module Crystal
end
end

macro trace(section, operation, **metadata, &block)
\{{yield}}
def self.trace(section : Tracing::Section, operation : String, time : UInt64? = nil, **metadata, &)
yield
end

def self.trace(section : Tracing::Section, operation : String, time : UInt64? = nil, **metadata)
end
{% end %}
end
8 changes: 4 additions & 4 deletions src/gc/boehm.cr
Original file line number Diff line number Diff line change
Expand Up @@ -170,21 +170,21 @@

# :nodoc:
def self.malloc(size : LibC::SizeT) : Void*
Crystal.trace :gc, :malloc, size: size do
Crystal.trace :gc, "malloc", size: size do
LibGC.malloc(size)
end
end

# :nodoc:
def self.malloc_atomic(size : LibC::SizeT) : Void*
Crystal.trace :gc, :malloc, size: size, atomic: 1 do
Crystal.trace :gc, "malloc", size: size, atomic: 1 do
LibGC.malloc_atomic(size)
end
end

# :nodoc:
def self.realloc(ptr : Void*, size : LibC::SizeT) : Void*
Crystal.trace :gc, :realloc, size: size do
Crystal.trace :gc, "realloc", size: size do
LibGC.realloc(ptr, size)
end
end
Expand Down Expand Up @@ -230,7 +230,7 @@
@@on_heap_resize = LibGC.get_on_heap_resize

LibGC.set_on_heap_resize(->(new_size : LibGC::Word) {
Crystal.trace :gc, :heap_resize, size: new_size
Crystal.trace :gc, "heap_resize", size: new_size
@@on_heap_resize.try(&.call(new_size))
})
end
Expand Down Expand Up @@ -280,7 +280,7 @@
end

def self.free(pointer : Void*) : Nil
Crystal.trace :gc, :free do

Check failure on line 283 in src/gc/boehm.cr

View workflow job for this annotation

GitHub Actions / LLVM 13.0.0

expected argument #1 to 'Crystal.trace' to match a member of enum Crystal::Tracing::Section.

Check failure on line 283 in src/gc/boehm.cr

View workflow job for this annotation

GitHub Actions / LLVM 16.0.3

expected argument #1 to 'Crystal.trace' to match a member of enum Crystal::Tracing::Section.

Check failure on line 283 in src/gc/boehm.cr

View workflow job for this annotation

GitHub Actions / LLVM 18.1.4

expected argument #1 to 'Crystal.trace' to match a member of enum Crystal::Tracing::Section.

Check failure on line 283 in src/gc/boehm.cr

View workflow job for this annotation

GitHub Actions / LLVM 14.0.0

expected argument #1 to 'Crystal.trace' to match a member of enum Crystal::Tracing::Section.

Check failure on line 283 in src/gc/boehm.cr

View workflow job for this annotation

GitHub Actions / LLVM 15.0.6

expected argument #1 to 'Crystal.trace' to match a member of enum Crystal::Tracing::Section.

Check failure on line 283 in src/gc/boehm.cr

View workflow job for this annotation

GitHub Actions / LLVM 17.0.6

expected argument #1 to 'Crystal.trace' to match a member of enum Crystal::Tracing::Section.
LibGC.free(pointer)
end
end
Expand Down
8 changes: 4 additions & 4 deletions src/gc/none.cr
Original file line number Diff line number Diff line change
Expand Up @@ -9,19 +9,19 @@ module GC

# :nodoc:
def self.malloc(size : LibC::SizeT) : Void*
Crystal.trace :gc, :malloc, size: size
Crystal.trace :gc, "malloc", size: size
LibC.malloc(size)
end

# :nodoc:
def self.malloc_atomic(size : LibC::SizeT) : Void*
Crystal.trace :gc, :malloc, size: size, atomic: 1
Crystal.trace :gc, "malloc", size: size, atomic: 1
LibC.malloc(size)
end

# :nodoc:
def self.realloc(pointer : Void*, size : LibC::SizeT) : Void*
Crystal.trace :gc, :realloc, size: size
Crystal.trace :gc, "realloc", size: size
LibC.realloc(pointer, size)
end

Expand All @@ -35,7 +35,7 @@ module GC
end

def self.free(pointer : Void*) : Nil
Crystal.trace :gc, :free
Crystal.trace :gc, "free"
LibC.free(pointer)
end

Expand Down
Loading