summaryrefslogtreecommitdiff
path: root/Documentation/tools
AgeCommit message (Collapse)Author
2025-12-05Merge tag 'trace-tools-v6.19' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace Pull rtla trace tooling updates from Steven Rostedt: - Officially add Tomas Glozar as a maintainer to RTLA tool - Add for_each_monitored_cpu() helper In multiple places, RTLA tools iterate over the list of CPUs running tracer threads. Use single helper instead of repeating the for/if combination. - Remove unused variable option_index in argument parsing RTLA tools use getopt_long() for argument parsing. For its last argument, an unused variable "option_index" is passed. Remove the variable and pass NULL to getopt_long() to shorten the naturally long parsing functions, and make them more readable. - Fix unassigned nr_cpus after code consolidation In recent code consolidation, timerlat tool cleanup, previously implemented separately for each tool, was moved to a common function timerlat_free(). The cleanup relies on nr_cpus being set. This was not done in the new function, leaving the variable uninitialized. Initialize the variable properly, and remove silencing of compiler warning for uninitialized variables. - Stop tracing on user latency in BPF mode Despite the name, rtla-timerlat's -T/--thread option sets timerlat's stop_tracing_total_us option, which also stops tracing on return-from-user latency, not only on thread latency. Implement the same behavior also in BPF sample collection stop tracing handler to avoid a discrepancy and restore correspondence of behavior with the equivalent option of cyclictest. - Fix threshold actions always triggering A bug in threshold action logic caused the action to execute even if tracing did not stop because of threshold. Fix the logic to stop correctly. - Fix few minor issues in tests Extend tests that were shown to need it to 5s, fix osnoise test calling timerlat by mistake, and use new, more reliable output checking in timerlat's "top stop at failed action" test. - Do not print usage on argument parsing error RTLA prints the entire usage message on encountering errors in argument parsing, like a malformed CPU list. The usage message has gotten too long. Instead of printing it, use newly added fatal() helper function to simply exit with the error message, excluding the usage. - Fix unintuitive -C/--cgroup interface "-C cgroup" and "--cgroup cgroup" are invalid syntax, despite that being a common way to specify an option with argument. Moreover, using them fails silently and no cgroup is set. Create new helper function to unify the handling of all such options and allow all of: -Xsomething -X=something -X something as well as the equivalent for the long option. - Fix -a overriding -t argument filename Fix a bug where -a following -t custom_file.txt overrides the custom filename with the default timerlat_trace.txt. - Stop tracing correctly on multiple events at once In some race scenarios, RTLA BPF sample collection might send multiple stop tracing events via the BPF ringbuffer at once. Compare the number of events for != 0 instead of == 1 to cover for this scenario and stop tracing properly. * tag 'trace-tools-v6.19' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: rtla/timerlat: Exit top main loop on any non-zero wait_retval rtla/tests: Don't rely on matching ^1ALL rtla: Fix -a overriding -t argument rtla: Fix -C/--cgroup interface tools/rtla: Replace osnoise_hist_usage("...") with fatal("...") tools/rtla: Replace osnoise_top_usage("...") with fatal("...") tools/rtla: Replace timerlat_hist_usage("...") with fatal("...") tools/rtla: Replace timerlat_top_usage("...") with fatal("...") tools/rtla: Add fatal() and replace error handling pattern rtla/tests: Fix osnoise test calling timerlat rtla/tests: Extend action tests to 5s tools/rtla: Fix --on-threshold always triggering rtla/timerlat_bpf: Stop tracing on user latency tools/rtla: Fix unassigned nr_cpus tools/rtla: Remove unused optional option_index tools/rtla: Add for_each_monitored_cpu() helper MAINTAINERS: Add Tomas Glozar as a maintainer to RTLA tool
2025-11-21rtla: Fix -C/--cgroup interfaceIvan Pravdin
Currently, user can only specify cgroup to the tracer's thread the following ways: `-C[cgroup]` `-C[=cgroup]` `--cgroup[=cgroup]` If user tries to specify cgroup as `-C [cgroup]` or `--cgroup [cgroup]`, the parser silently fails and rtla's cgroup is used for the tracer threads. To make interface more user-friendly, allow user to specify cgroup in the aforementioned way, i.e. `-C [cgroup]` and `--cgroup [cgroup]`. Refactor identical logic between -t/--trace and -C/--cgroup into a common function. Change documentation to reflect this user interface change. Fixes: a957cbc02531 ("rtla: Add -C cgroup support") Signed-off-by: Ivan Pravdin <ipravdin.official@gmail.com> Reviewed-by: Tomas Glozar <tglozar@redhat.com> Link: https://lore.kernel.org/r/16132f1565cf5142b5fbd179975be370b529ced7.1762186418.git.ipravdin.official@gmail.com [ use capital letter in subject, as required by tracing subsystem ] Signed-off-by: Tomas Glozar <tglozar@redhat.com>
2025-11-05Documentation/rtla: Include defaults for tracer optionsTomas Glozar
Commit 0122938a7ab4 ("rtla: Always set all tracer options") changed the behavior of RTLA to always set all osnoise and timerlat tracer options to default values taken from the tracers whenever an RTLA measurement is started. The change was done to make RTLA results consistent on subsequent runs of the same command. Include the default values for tracer options also in documentation where appropriate. Signed-off-by: Tomas Glozar <tglozar@redhat.com> Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <20251010083338.478961-10-tglozar@redhat.com>
2025-11-05Documentation/rtla: Mention default cgroup stateTomas Glozar
The RTLA option -C/--cgroup is used to set a cgroup for workload threads. This is either a specific cgroup, if passed an argument, or rtla's cgroup, if no argument is given. Expand the documentation of the -C option to also include the information about the cgroup settings when the option is not specified. Signed-off-by: Tomas Glozar <tglozar@redhat.com> Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <20251010083338.478961-8-tglozar@redhat.com>
2025-11-05Documentation/rtla: Mention default priorityTomas Glozar
RTLA allows the priority of workload threads to be set using the -P option. This is covered in docs, but the default state for RTLA's own user workload (implemented in timerlat_u.c) is not mentioned. Add mention of the default user workload priority as well as a reference to osnoise and timerlat tracers for kernel workload priority. Signed-off-by: Tomas Glozar <tglozar@redhat.com> Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <20251010083338.478961-7-tglozar@redhat.com>
2025-11-05Documentation/rtla: Correct tracer name for common optionsTomas Glozar
Several options in common_options.rst say "osnoise tracer" for both osnoise and timerlat. Use |tool| variable so that the correct tool name is used. Fixes: b1be48307de4 ("rtla: Add rtla osnoise top documentation") Signed-off-by: Tomas Glozar <tglozar@redhat.com> Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <20251010083338.478961-6-tglozar@redhat.com>
2025-11-05Documentation/rtla: Fix typo in common_timerlat_options.txtTomas Glozar
Fix spelling error "equilavent" in place of "equivalent". Fixes: 173a3b014827 ("rtla/timerlat: Add the automatic trace option") Signed-off-by: Tomas Glozar <tglozar@redhat.com> Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <20251010083338.478961-5-tglozar@redhat.com>
2025-11-05Documentation/rtla: Fix typo in rtla-timerlat-top.rstTomas Glozar
Fix "seem" in place of intended "seen" in rtla-timerlat-top documentation. Fixes: df337d014b57 ("rtla: Add rtla timerlat top documentation") Signed-off-by: Tomas Glozar <tglozar@redhat.com> Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <20251010083338.478961-4-tglozar@redhat.com>
2025-11-05Documentation/rtla: Fix typo in common_timerlat_options.txtTomas Glozar
Fix "awakes" being used in place of "awakened" in --users-threads option documentation. Fixes: 612738321774 ("Documentation: Add tools/rtla timerlat -u option documentation") Signed-off-by: Tomas Glozar <tglozar@redhat.com> Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <20251010083338.478961-3-tglozar@redhat.com>
2025-11-05Documentation/rtla: Fix typo in common_options.txtTomas Glozar
Fix "unlike" being spelled "nlike" in --on-threshold documentation. Fixes: 70165c78e31d ("Documentation/rtla: Add actions feature") Signed-off-by: Tomas Glozar <tglozar@redhat.com> Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <20251010083338.478961-2-tglozar@redhat.com>
2025-10-17Documentation/rtla: rename common_xxx.rst files to common_xxx.txtGopi Krishna Menon
Sphinx reports htmldocs errors: Documentation/tools/rtla/common_options.rst:58: ERROR: Undefined substitution referenced: "threshold". Documentation/tools/rtla/common_options.rst:88: ERROR: Undefined substitution referenced: "tool". Documentation/tools/rtla/common_options.rst:88: ERROR: Undefined substitution referenced: "thresharg". Documentation/tools/rtla/common_options.rst:88: ERROR: Undefined substitution referenced: "tracer". Documentation/tools/rtla/common_options.rst:92: ERROR: Undefined substitution referenced: "tracer". Documentation/tools/rtla/common_options.rst:98: ERROR: Undefined substitution referenced: "actionsperf". Documentation/tools/rtla/common_options.rst:113: ERROR: Undefined substitution referenced: "tool". common_*.rst files are snippets that are intended to be included by rtla docs (rtla*.rst). common_options.rst in particular contains substitutions which depend on other common_* includes, so building it independently as reST source results in above errors. Rename all common_*.rst files to common_*.txt to prevent Sphinx from building these snippets as standalone reST source and update all include references accordingly. Link: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#substitutions Suggested-by: Tomas Glozar <tglozar@redhat.com> Suggested-by: Bagas Sanjaya <bagasdotme@gmail.com> Signed-off-by: Gopi Krishna Menon <krishnagopi487@gmail.com> Reviewed-by: Tomas Glozar <tglozar@redhat.com> Fixes: 05b7e10687c6 ("tools/rtla: Add remaining support for osnoise actions") Reviewed-by: Bagas Sanjaya <bagasdotme@gmail.com> Link: https://lore.kernel.org/r/20251008184522.13201-1-krishnagopi487@gmail.com [Bagas: massage commit message and apply trailers] Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com> Reviewed-by: Randy Dunlap <rdunlap@infradead.org> Tested-by: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Message-ID: <20251013092719.30780-2-bagasdotme@gmail.com>
2025-09-27tools/rtla: Add remaining support for osnoise actionsCrystal Wood
The basic functionality came with the consolidation; now hook up the command line options, and add documentation and tests. Cc: John Kacur <jkacur@redhat.com> Cc: Costa Shulyupin <costa.shul@redhat.com> Link: https://lore.kernel.org/20250907022325.243930-8-crwood@redhat.com Reviewed-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Crystal Wood <crwood@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-08-01Merge tag 'trace-tools-v6.17' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace Pull tracing tools updates from Steven Rostedt: - Introduce enum timerlat_tracing_mode Now that BPF based sampling has been added to timerlat, add an enum to represent which mode timerlat is running in - Add action on timelat threshold feature A new option, --on-threshold, is added, taking an argument that further specifies the action. Actions added in this patch are: - trace[,file=<filename>]: Saves tracefs buffer, optionally taking a filename - signal,num=<sig>,pid=<pid>: Sends signal to process. "parent" might be specified instead of number to send signal to parent process - shell,command=<command>: Execute shell command - Allow resuming tracing in timerlat bpf rtla-timerlat BPF program uses a global variable stored in a .bss section to store whether tracing has been stopped. Map it to allow it to resume tracing after it has been stopped - Add continue action to timerlat Introduce option to resume tracing after a latency threshold overflow. The option is implemented as an action named "continue" - Add action on end feature to timerlat Implement actions on end next to actions on threshold. A new option, --on-end is added, parallel to --on-threshold. Instead of being executed whenever a latency threshold is reached, it is executed at the end of the measurement - Have rtla tests check output with grep Add argument to the check command in the test suite that takes a regular expression that the output of rtla command is checked against. This allows testing for specific information in rtla output in addition to checking the return value - Add tests for timerlat actions - Update the documentation for the new features * tag 'trace-tools-v6.17' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: rtla/tests: Test timerlat -P option using actions rtla/tests: Add grep checks for base test cases Documentation/rtla: Add actions feature rtla/tests: Limit duration to maximum of 10s rtla/tests: Add tests for actions rtla/tests: Check rtla output with grep rtla/timerlat: Add action on end feature rtla/timerlat: Add continue action rtla/timerlat_bpf: Allow resuming tracing rtla/timerlat: Add action on threshold feature rtla/timerlat: Introduce enum timerlat_tracing_mode
2025-07-25Documentation/rtla: Add actions featureTomas Glozar
Document both --on-threshold and --on-end, with examples. Cc: John Kacur <jkacur@redhat.com> Cc: Luis Goncalves <lgoncalv@redhat.com> Cc: Arnaldo Carvalho de Melo <acme@kernel.org> Cc: Chang Yin <cyin@redhat.com> Cc: Costa Shulyupin <costa.shul@redhat.com> Cc: Crystal Wood <crwood@redhat.com> Cc: Gabriele Monaco <gmonaco@redhat.com> Link: https://lore.kernel.org/20250626123405.1496931-10-tglozar@redhat.com Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-07-17Documentation/rtla: Describe exit statusCosta Shulyupin
Commit 18682166f61494072d58 ("rtla: Set distinctive exit value for failed tests") expands exit status making it useful. Add section 'EXIT STATUS' and required SPDX-License-Identifier to the documentation. Signed-off-by: Costa Shulyupin <costa.shul@redhat.com> Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org> [jc: fixed sphinx error caused by missing blank line] Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250608105531.758809-2-costa.shul@redhat.com
2025-07-17Documentation/rtla: Add include common_appendix.rstCosta Shulyupin
Add include common_appendix.rst into Documentation/tools/rtla/rtla-timerlat-hist.rst - the only file of rtla-*.rst still without common_appendix.rst. Signed-off-by: Costa Shulyupin <costa.shul@redhat.com> Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250608104437.753708-2-costa.shul@redhat.com
2025-04-14Documentation/rtla: Include BPF sample collectionTomas Glozar
Add dependencies needed to build rtla with BPF sample collection support to README, and document both ways of sample collection in the manpages. Signed-off-by: Tomas Glozar <tglozar@redhat.com> Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org> Reviewed-by: Luis Claudio R. Goncalves <lgoncalv@redhat.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250311114936.148012-5-tglozar@redhat.com
2025-04-14Documentation/rtla: Fix typo in common_timerlat_description.rstTomas Glozar
Fix "it enable" to "it enables". Fixes: 29380d4055e5 ("rtla: Add rtla timerlat documentation") Signed-off-by: Tomas Glozar <tglozar@redhat.com> Reviewed-by: Luis Claudio R. Goncalves <lgoncalv@redhat.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250311114936.148012-4-tglozar@redhat.com
2025-04-14Documentation/rtla: Fix typo in rtla-timerlat.rstTomas Glozar
The file says "rtla hist hist mode" instead of "rtla timerlat hist mode". Fix the typo. Fixes: 29380d4055e5 ("rtla: Add rtla timerlat documentation") Signed-off-by: Tomas Glozar <tglozar@redhat.com> Reviewed-by: Luis Claudio R. Goncalves <lgoncalv@redhat.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250311114936.148012-3-tglozar@redhat.com
2025-04-14Documentation/rtla: Fix duplicate text about timerlat tracerTomas Glozar
A passage about how the timerlat tracer outputs information is included in both common_timerlat_description.rst and rtla-timerlat.rst, leading it to be displayed twice in the rtla-timerlat page. Remove the duplicate passage from rtla-timerlat.rst. Fixes: 29380d4055e5 ("rtla: Add rtla timerlat documentation") Signed-off-by: Tomas Glozar <tglozar@redhat.com> Reviewed-by: Luis Claudio R. Goncalves <lgoncalv@redhat.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250311114936.148012-2-tglozar@redhat.com
2025-03-27Documentation/rv: Add sched pages to the indicesGabriele Monaco
The pages Documentation/tools/rv/rv-mon-sched.rst and Documentation/trace/rv/monitor_sched.rst were introduced but not included in any index. Add them to the respective indices. Cc: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/20250327081240.46422-1-gmonaco@redhat.com Reported-by: Stephen Rothwell <sfr@canb.auug.org.au> Fixes: 03abeaa63c08 ("Documentation/rv: Add docs for the sched monitors") Signed-off-by: Gabriele Monaco <gmonaco@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-03-24Documentation/rv: Add docs for the sched monitorsGabriele Monaco
Add man page and kernel documentation for the sched monitors, as sched is a container of other monitors, document all in the same page. sched is the first nested monitor, also explain what is a nested monitor and how enabling containers or children monitors work. To: Ingo Molnar <mingo@redhat.com> To: Peter Zijlstra <peterz@infradead.org> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Jonathan Corbet <corbet@lwn.net> Cc: John Kacur <jkacur@redhat.com> Cc: Clark Williams <williams@redhat.com> Link: https://lore.kernel.org/20250305140406.350227-9-gmonaco@redhat.com Signed-off-by: Gabriele Monaco <gmonaco@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2024-10-17rtla: Documentation: Mention --deepest-idle-stateTomas Glozar
Add --deepest-idle-state to manpage and mention libcpupower dependency in README.txt. Link: https://lore.kernel.org/20241017140914.3200454-7-tglozar@redhat.com Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2024-07-03Documentation/tools/rv: fix document headerJiri Kastner
align header of document with filename and rest of the content Signed-off-by: Jiri Kastner <cz172638@gmail.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20240626203906.191841-1-cz172638@gmail.com
2024-05-16rtla: Documentation: Fix -t, --traceJohn Kacur
Move -t, --trace from common_options.rst to common_osnoise_options.rst and common_timerlat_options.rst so that it will appear in the man pages rtla-timerlat-hist.1 rtla-timerlat-top.1 rtla-osnoise-hist.1 rtla-osnoise-top.1 Remove the equals ('=') sign and add a space. Link: https://lkml.kernel.org/r/20240516143121.12614-1-jkacur@redhat.com Cc: Daniel Bristot de Oliveria <bristot@kernel.org> Signed-off-by: John Kacur <jkacur@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
2024-05-16rtla: Add --trace-buffer-size optionDaniel Bristot de Oliveira
Add the option allow the users to set a different buffer size for the trace. For example, in large systems, the user might be interested on reducing the trace buffer to avoid large tracing files. The buffer size is specified in kB, and it is only affecting the tracing instance. The function trace_set_buffer_size() appears on libtracefs v1.6, so increase the minimum required version on Makefile.config. Link: https://lkml.kernel.org/r/e7c9ca5b3865f28e131a49ec3b984fadf2d056c6.1715860611.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: John Kacur <jkacur@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
2024-05-15rtla/timerlat: Make user-space threads the defaultDaniel Bristot de Oliveira
After ther -u addition, most of the known users are setting it. And it makes sense, as it adds more information, and inherits the default setup for the threads - e.g., cgroups configs. Thus, if the user-space interface is available, enable -u. Otherwise, use the in-kernel thread. Add the -k option to allow the user to request kernel-threads. Link: https://lkml.kernel.org/r/9241d3089de4091b124f780ed832a0e6646cadaa.1713968967.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Cc: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
2024-05-15rtla: Add the --warm-up optionDaniel Bristot de Oliveira
On many cases, the results right after the startup are different from the rest of the execution, biasing the results. For example, on osnoise, the scheduler might take some time to adapt to the new busy-loop workload. Add the --warm-up <seconds> option, adding a warm-up phase (in seconds) where the workload is set, but the results are discarded. Link: https://lkml.kernel.org/r/e682d5ce5af90f123bd13220f63d5c3d118a92be.1713968967.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Cc: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
2024-03-20tools/rtla: Add -U/--user-load option to timerlatDaniel Bristot de Oliveira
The timerlat tracer provides an interface for any application to wait for the timerlat's periodic wakeup. Currently, rtla timerlat uses it to dispatch its user-space workload (-u option). But as the tracer interface is generic, rtla timerlat can also be used to monitor any workload that uses it. For example, a user might place their own workload to wait on the tracer interface, and monitor the results with rtla timerlat. Add the -U option to rtla timerlat top and hist. With this option, rtla timerlat will not dispatch its workload but only setting up the system, waiting for a user to dispatch its workload. The sample code in this patch is an example of python application that loops in the timerlat tracer fd. To use it, dispatch: # rtla timerlat -U In a terminal, then run the python program on another terminal, specifying the CPU to run it. For example, setting on CPU 1: #./timerlat_load.py 1 Then rtla timerlat will start printing the statistics of the ./timerlat_load.py app. An interesting point is that the "Ret user Timer Latency" value is the overall response time of the load. The sample load does a memory copy to exemplify that. The stop tracing options on rtla timerlat works in this setup as well, including auto analysis. Link: https://lkml.kernel.org/r/36e6bcf18fe15c7601048fd4c65aeb193c502cc8.1707229706.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Cc: Masami Hiramatsu <mhiramat@kernel.org> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
2023-09-22rtla: fix a example in rtla-timerlat-hist.rstXie XiuQi
The following error message is reported when running the example in document. # timerlat hist -d 10m -c 0-4 -P d:100us:1ms -p 1ms --no-aa Failed to set timerlat period Could not apply config The unit of the period is microsecond, '1ms' cannot be accepted. usage: [rtla] timerlat hist [-h] [-q] [-d s] [-D] [-n] [-a us] [-p us] [-i us] [-T us] [-s us] ... ... -p/--period us: timerlat period in us ... Also fix another minor missleading comment. Link: https://lore.kernel.org/lkml/20230919133028.697144-1-xiexiuqi@huaweicloud.com Signed-off-by: Xie XiuQi <xiexiuqi@huawei.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org>
2023-08-18Documentation: Fix typosBjorn Helgaas
Fix typos in Documentation. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Link: https://lore.kernel.org/r/20230814212822.193684-4-helgaas@kernel.org Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2023-06-13Documentation: Add tools/rtla timerlat -u option documentationDaniel Bristot de Oliveira
Add the -u/--user-thread option documentation for timerlat top/hist. Link: https://lkml.kernel.org/r/bf727c8ccb8f50792200ae620141e047edf4af7a.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-06-13rtla/timerlat_hist: Add auto-analysis supportDaniel Bristot de Oliveira
Add auto-analysis to timerlat hist, including the --no-aa option to reduce overhead and --dump-task. --aa-only was not added as it is already on timerlat top. Link: https://lkml.kernel.org/r/c2693f47ee83e659a7723fed8035f5d2534f528e.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-06-13rtla: Add --house-keeping optionDaniel Bristot de Oliveira
To avoid having rtla interfering with the measurement threads, add an option for the user to set the CPUs in which rtla should run. For instance: # rtla timerlat top -H 0 -c 1-7 Will place rtla in the CPU 0, while running the measurement threads in the CPU 1-7. Link: https://lkml.kernel.org/r/6a6c78a579a96ba8b02ae67ee1e0ba2cb5e03c4a.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Suggested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-06-13rtla: Add -C cgroup supportDaniel Bristot de Oliveira
The -C option sets a cgroup to the tracer's threads. If the -C option is passed without arguments, the tracer's thread will inherit rtla's cgroup. Otherwise, the threads will be placed on the cgroup passed to the option. Link: https://lkml.kernel.org/r/cb051477331d292f17c08bf1d66f0e0384bbe5a5.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-25rtla/timerlat: Add auto-analysis only optionDaniel Bristot de Oliveira
Parsing and formating timerlat data might consume a reasonable amount of CPU time on very large systems, or when timerlat has a too short period. Add an option to run timerlat with auto-analysis enabled while skipping the statistics parsing. In this mode, rtla timerlat periodically checks if the tracing is on, going to sleep waiting for the stop tracing condition to stop tracing, or for the tracing session to finish. If the stop tracing condition is hit, the tool prints the auto analysis. Otherwise, the tool prints the max observed latency and exit. The max observed latency is captured via tracing_max_latency. Link: https://lore.kernel.org/linux-trace-devel/4dc514d1d5dc353c537a466a9b5af44c266b6da2.1680106912.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-02-13Documentation/rtla: Add hwnoise man pageDaniel Bristot de Oliveira
Add a man page for the new rtla hwnoise tool, mostly based in the rtla osnoise top. Link: https://lkml.kernel.org/r/27088e0f42a6129e9b75f789d78adff4171f2e79.1675805361.git.bristot@kernel.org Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Reviewed-by: Bagas Sanjaya <bagasdotme@gmail.com> Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Clark Williams <williams@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-02-02Documentation/rtla: Add timerlat-top auto-analysis optionsDaniel Bristot de Oliveira
Add the new options to the man page, as well as updating the example to include the new output. Link: https://lkml.kernel.org/r/a3f5fb49432a55c3323b18725fc6e702f16ccc79.1675179318.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-12-09Documentation/rv: Add verification/rv man pagesDaniel Bristot de Oliveira
Add man pages for the rv command line, using the same scheme we used in rtla. Link: https://lkml.kernel.org/r/e841d7cfbdfc3ebdaf7cbd40278571940145d829.1668180100.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-10-13Merge tag 'docs-6.1-2' of git://git.lwn.net/linuxLinus Torvalds
Pull documentation fixes from Jonathan Corbet: "A handful of relatively simple documentation fixes, plus a set of patches catching the Chinese translation up with the front-page rework" * tag 'docs-6.1-2' of git://git.lwn.net/linux: Documentation: rtla: Correct command line example docs/zh_CN: add a man-pages link to zh_CN/index.rst docs/zh_CN: Rewrite the Chinese translation front page docs/zh_CN: add zh_CN/arch.rst docs/zh_CN: promote the title of zh_CN/process/index.rst docs/zh_CN: Update the translation of page_owner to 6.0-rc7 docs/zh_CN: Update the translation of ksm to 6.0-rc7 docs/howto: Replace abundoned URL of gmane.org Documentation: ubifs: Fix compression idiom Documentation/mm/page_owner.rst: delete frequently changing experimental data docs/zh_CN: Fix build warning docs: ftrace: Correct access mode
2022-10-13Documentation: rtla: Correct command line examplePierre Gondois
The '-t/-T' parameters seem to have been swapped: -t/--trace[=file]: save the stopped trace to [file|timerlat_trace.txt] -T/--thread us: stop trace if the thread latency is higher than the argument in us Swap them back. Signed-off-by: Pierre Gondois <pierre.gondois@arm.com> Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org> Link: https://lore.kernel.org/r/20221006084409.3882542-1-pierre.gondois@arm.com Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2022-08-10rtla: Fix tracer nameAlexandre Vicenzi
The correct tracer name is timerlat and not timelat. Link: https://lore.kernel.org/linux-trace-devel/20220808180343.22262-1-alexandre.vicenzi@suse.com Signed-off-by: Alexandre Vicenzi <alexandre.vicenzi@suse.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-05-29Merge tag 'trace-tools-v5.19' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace Pull tracing tool updates from Steven Rostedt: - Various clean ups and fixes to rtla (Real Time Linux Analysis) * tag 'trace-tools-v5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace: rtla: Remove procps-ng dependency rtla: Fix __set_sched_attr error message rtla: Minor grammar fix for rtla README rtla: Don't overwrite existing directory mode rtla: Avoid record NULL pointer dereference rtla/Makefile: Properly handle dependencies
2022-05-26rtla/Makefile: Properly handle dependenciesDaniel Bristot de Oliveira
Linus had a problem compiling RTLA, saying: "[...] I wish the tracing tools would do a bit more package checking and helpful error messages too, rather than just fail with: fatal error: tracefs.h: No such file or directory" Which is indeed not a helpful message. Update the Makefile, adding proper checks for the dependencies, with useful information about how to resolve possible problems. For example, the previous error is now reported as: $ make ******************************************** ** NOTICE: libtracefs version 1.3 or higher not found ** ** Consider installing the latest libtracefs from your ** distribution, e.g., 'dnf install libtracefs' on Fedora, ** or from source: ** ** https://git.kernel.org/pub/scm/libs/libtrace/libtracefs.git/ ** ******************************************** These messages are inspired by the ones used on trace-cmd, as suggested by Stevel Rostedt. Link: https://lore.kernel.org/r/CAHk-=whxmA86E=csNv76DuxX_wYsg8mW15oUs3XTabu2Yc80yw@mail.gmail.com/ Changes from V1: - Moved the rst2man check to the install phase (when it is used). - Removed the procps-ng lib check [1] as it is being removed. [1] a0f9f8c1030c66305c9b921057c3d483064d5529.1651220820.git.bristot@kernel.org Link: https://lkml.kernel.org/r/3f1fac776c37e4b67c876a94e5a0e45ed022ff3d.1651238057.git.bristot@kernel.org Cc: Ingo Molnar <mingo@kernel.org> Cc: Andrew Morton <akpm@linux-foundation.org> Reported-by: Linus Torvalds <torvalds@linux-foundation.org> Suggested-by: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-04-20rtla: Documentation: fix email addressesRandy Dunlap
Correct lkml@vger to linux-kernel@vger. Add linux-trace-devel since that is what is in the MAINTAINERS file. Fixes: d40d48e1f1f2 ("rtla: Add Documentation") Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2022-03-23Merge tag 'trace-rtla-v5.18' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace Pull RTLA tracing tool updates from Steven Rostedt: "Real Time Analysis Tool updatesfor 5.18: - Support for adjusting tracing_threashold - Add -a (auto) option to make it easier for users to debug in the field - Add -e option to add more events to the trace - Add --trigger option to add triggers to events - Add --filter option to filter events - Add support to save histograms to the file - Add --dma-latency to set /dev/cpu_dma_latency - Other fixes and cleanups" * tag 'trace-rtla-v5.18' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace: rtla: Tools main loop cleanup rtla/timerlat: Add --dma-latency option rtla/osnoise: Fix osnoise hist stop tracing message rtla: Check for trace off also in the trace instance rtla/trace: Save event histogram output to a file rtla: Add --filter support rtla/trace: Add trace event filter helpers rtla: Add --trigger support rtla/trace: Add trace event trigger helpers rtla: Add -e/--event support rtla/trace: Add trace events helpers rtla/timerlat: Add the automatic trace option rtla/osnoise: Add the automatic trace option rtla/osnoise: Add an option to set the threshold rtla/osnoise: Add support to adjust the tracing_thresh
2022-03-15rtla/timerlat: Add --dma-latency optionDaniel Bristot de Oliveira
Add the --dma-latency to set /dev/cpu_dma_latency to the specified value, this aims to avoid having exit from idle states latencies that could be influencing the analysis. Link: https://lkml.kernel.org/r/72ddb0d913459f13217086dadafad88a7c46dd28.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-03-15rtla/trace: Save event histogram output to a fileDaniel Bristot de Oliveira
The hist: trigger generates a histogram in the file sys/event/hist. If the hist: trigger is used, automatically save the histogram output of the event sys:event in the sys_event_hist.txt file. Link: https://lkml.kernel.org/r/b5c906af31d4e022ffe87fb0848fac5c089087c8.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-03-15rtla: Add --filter supportDaniel Bristot de Oliveira
Add --filter option. This option enables a trace event filtering of the previous -e sys:event argument. This option is available for all current tools. Link: https://lkml.kernel.org/r/509d70b6348d3e5bcbf1f07ab725ce08d063149a.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-03-15rtla: Add --trigger supportDaniel Bristot de Oliveira
Add --trigger option. This option enables a trace event trigger to the previous -e sys:event argument, allowing some advanced tracing options. For instance, in a system with CPUs 2:23 isolated, it is possible to get a stack trace of thread wakeup targeting those CPUs while running osnoise with the following command line: # osnoise top -c 2-23 -a 50 -e sched:sched_wakeup --trigger="stacktrace if target_cpu >= 2" This option is available for all current tools. Link: https://lkml.kernel.org/r/07d2983d5f71261d4da89dbaf02efcad100ab8ee.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>