Steadfast Update I - The Baseline

In the realm of software development, the importance of thoughtful consideration before diving into coding cannot be overstated. Rich Hickey, a renowned figure in the field, coined the term "Hammock-driven development" to describe the practice of dedicating ample time to contemplation and ideation before engaging in actual programming. This approach emphasizes the significance of understanding a problem thoroughly, weighing its consequences and alternatives, and exploring it in its entirety before taking action.

Throughout personal and professional experiences, it has become evident that a lack of thorough thinking before acting can lead to undesirable outcomes. The Hebrew adage "סוֹף מַעֲשֶׂה בְּמַחֲשָׁבָה תְּחִלָּה" encapsulates this sentiment, highlighting the significance of initial thought in determining the ultimate success of a plan.

To illustrate this principle, one might compare the ability to envision game moves several steps ahead to the process of contemplating a problem before implementing a solution. In the same vein, it would be highly beneficial if those in positions of leadership could adopt a similar mindset, allowing for more comprehensive and effective decision-making.

While the author does not often engage in hammock-based reflection, they do appreciate the value of contemplating code in various settings, such as while performing household chores, taking walks, driving, or even during moments of rest. This approach enables the mind to process ideas and generate insights in a more organic manner. Additionally, jotting down thoughts in a notebook can serve as a valuable tool in this process.

The author's latest open-source project, Steadfast, exemplifies the power of combining extensive contemplation with practical implementation. This project has been on the author's mind for a considerable period, and the key to its advancement has been the emphasis on both thinking and sketching out ideas. By dedicating a significant amount of time to contemplation, the author was able to identify critical problems, envision potential solutions, and develop a clear understanding of the project's architecture, API, and limitations.

In recent weeks, the author has been focused on laying the groundwork for Steadfast. One of the initial tasks involved implementing functionality to convert a given Prism Abstract Syntax Tree (AST) into source code. While this task has been accomplished in the past during the development of the Papercraft compiler, the author sought to enhance and streamline the process for Steadfast.

However, the implementation of this functionality in Ruby presents certain challenges. The Ruby runtime does not currently support passing an AST directly to the eval function. To overcome this limitation, the author recognized the need to convert Ruby AST nodes back to their underlying C structures, pm_node, and then feed them into the eval implementation.

This requires a deep understanding of the Ruby runtime and the Prism compiler, as well as careful consideration of the implications for exception handling and backtraces.

To address the issue of backtraces generated when evaluating a synthetic AST, the author also implemented a mechanism to track the original source location of each piece of the generated code. This is crucial for accurately reporting errors and understanding the source of exceptions raised during evaluation. The author achieved this by adding filename information to the Prism::Source class, an instance of which is associated with each AST node.

By leveraging this information, they were able to map generated source code back to its corresponding source locations.

Steadfast currently offers several tools to facilitate this process. The `Steadfast.to_ast(obj)` method converts a proc or method into its corresponding AST using the new AST APIs introduced in Ruby 4.1. Additionally, it sets a `__filename__` attribute for the underlying Prism::Source instance, enabling the tracking of source locations.

The author acknowledges the potential for providing alternative methods to accommodate older Ruby versions or situations where enabling `RubyVM.keep_script_lines` is not feasible, but these features are currently under consideration.

Another tool provided by Steadfast is `Steadfast.to_source(obj)`, which generates source code from a given proc, method, or AST. The generated code, while not optimized for readability, focuses on correctness, testability, and the ability to accurately translate exception backtraces for composed ASTs, which may contain nodes from different source locations. The code generation algorithm prioritizes these aspects over formatting, as the primary goal is to ensure accurate representation of the original code.

Lastly, Steadfast includes the `Steadfast.translate_backtrace(bt)` method, which takes an exception backtrace and maps it back to the original source locations. This functionality is essential for properly diagnosing and resolving issues that arise during evaluation, as it provides context and clarity regarding the location of the problematic code.

In summary, the author's work on Steadfast demonstrates the value of taking a thoughtful, contemplative approach to software development. By dedicating time to understanding the problem at hand, exploring potential solutions, and carefully considering the implications of various approaches, the author has been able to create a tool that addresses the challenges of converting code and handling backtraces effectively.

As the project continues to evolve, it is likely that Steadfast will play an increasingly important role in the Ruby community, empowering developers to create more robust and reliable software.