*! version 0.0.0.9000 dec 15 2024
cache -- A program to cache all other Stata commands
https://github.com/randrescastaneda/cache/

Syntax

cache
[subcommand],
[options]
[: command]

Subcommand

clean Cleans all previously cached commands and any saved elements.

list Lists all currently cached commands

dir(string) Specifies the directory where cached contents of commands
will be saved to be restored later; default is dir(_cache)

project(string) Specifies a sub-directory within dir
if further organisation of cached commands is desired

prefix(string) Defines a prefix for all saved elements of a command;
default is _ch

nodata does not cache data if changes occur in data

clear allows command to proceed even if this saves over data currently in memory

replace Re-runs command and saves over previously cached version

keepall Does not clear previous ereturn and sreturn lists, permitting future use

command is any native or user written command available in Stata.

Description

The cache command allows for all output and returned elements of any
commands to be saved (cached), and reloaded in the future without re-running the
command. When cache is used, it will check if the indicated command has been
previously issued and cached by the user, and if so reload all elements returned
by the command, along with command output, without re-running the command itself.
Otherwise, if no previously cached result for the command exists,
cache will run the command, and cache all output and returns for future uses.

cache is useful if slow or resource-intensive commands are run more than once,
as after the first time they are run all output can be simply accessed from the
previously saved version, saving on all processing time. cache works with all valid
Stata commands and is issued as a prefix before the desired command.

cache stores all command output for the future, or loads all elements returned
or otherwise altered by the command including (where relevant): elements accessible after
the command in the ereturn, sreturn or return lists; graphical output; any alterations
to data (unless nodata is specified); any alterations to frames
(unless nodata is specified).

Remarks

One of either

cache subcommand [, options]

or

cache [, options] : command

should be issued. The use of subcommands is for general management
of cache and should not be combined with individual commands to be cached. Otherwise
the standard usage of cache: command will cache or
load the output of command.

Subcommands

clean Indicates that all contents of cached commands should be permanently removed from
the cache directory. Typing cache clean will result in a confirmation message to which
the user must respond "y" if all contents of the cache directory should be cleared. This option
should be used with care, as any contents of the cache directory will be permanently deleted.
To avoid issues with potential loss of important information, the clean sub-command does
NOT work recursively. For example, if various projects are located within a main cache subfolder,
these must be cleaned individually by using the dir() and/or project() options.

list Lists the full history of cached commands available for re-loading in the current
cache directory.

Options

dir(string) Specifies the directory where cached contents of commands will be saved to be
restored later. If not specified, a subdirectory of the current working directory named _cache
is used by default.

project(string) Allows for sub-folders within the cache directory if further control of
cached contents is desired. This permits cached commands to be stored in groups of related
commands.

prefix(string) By default, all cached contents of a command will be saved with a prefix
of _ch followed by the hash of the command as typed, along with the data signature of data in
memory. The prefix option will replace _ch with the indicated string

nodata If nodata is specified, cache will save all command returns, but will not
save data if any changes in data are detected. This may be desired if data in memory is
very large, and the command issued makes innocuous changes to the data that are not required
in the future. Note that by default, any regression-based commands will save resulting data,
given that a variable is generated to store information about the estimation sample via
e(sample). Thus, care should be taken if nodata is used with regression-based commands,
as the use of e(sample) will not be available.

This option should be used with care, as in future calls to cache all command
output and results will be returned, but updates to data will not be produced.

clear Allows command implementation to proceed even if this would unsaved changes in
data (similar, for example, to use, clear)

replace Forces cache to re-run the command and re-cache results, even if a
previously cached version of command output has been found. Such an example may be useful
if commands are re-issued and command behaviour has changed.

keepall Indicates that elements stored by previous commands in e(return) and s(return)
lists should not be cleared prior to invoking the command requested with cache.
By definition, cache will store all elements returned in ereturn, sreturn or return lists
following an issued command. The default usage is to clear ereturn, sreturn and return
lists prior to issuing the requested command, so that only elements from the issued command
will be returned in future calls to the command from cache. However, this implies that
elements of return lists not used by a specific command will not hold over from previous
commands, for example if an r-class command is issued, results from previous e-class commands
will be cleared to avoid being saved by cache. If such behaviour is undesired,
the keepall option should be used so that any previous return lists are maintained in
memory.

Examples

The examples below illustrate the idea of cache. These are based on
Stata's auto dataset.

Basic examples

Load Stata's auto dataset, and cache a command

. sysuse auto

. cache: regress price weight length

Now, inspect return list and ereturn list to see elements returned by the regress command

. return list

. ereturn list

Now, issue alternative command so that return lists will be altered

. cache: sum price weight length

Finally, call cache again, and confirm that cache has reloaded all original command output
without re-running the command:

. cache: regress price weight length

. return list

An example documenting time savings

Load Stata's auto dataset, set a timer, cache a command which will take considerable time to run
and then turn off the timer

. sysuse auto

. timer on 1

. cache: bootstrap, reps(5000) dots(100): reg price mpg

. timer off 1

Now, set a second timer and run the command from the cached version:

. timer on 2

. cache: bootstrap, reps(5000) dots(100): reg price mpg

. timer off 2

. timer list

Stored results

cache is itself an rclass command, and along with all
elements returned from the requested command, additionally stores the
following in r():

Macros

r(call_hash) The hash of the command as issued by the user

r(datasignature) The datasignature of the data prior to command being invoked

r(cmd_hash) The hash used by cache to uniquely identify the command

Authors

R.Andres Castaneda, The World Bank

Email: acastanedaa@worldbank.org

GitHub: randrescastaneda

Damian Clarke, Department of Economics, University of Chile & University of Exeter

Email: dclarke4@worldbank.org

Email: dclarke@fen.uchile.cl

GitHub: damiancclarke

Contact

Any comments, suggestions, or bugs can be reported in the
GitHub issues page.
All the files are available in the GitHub repository.

Thanks for citing this Stata command as follows

Castaneda, R.Andres and Damian Clarke. (2024)
"cache: Stata Module to to cache all other Stata commands"
(version 0.0.0.9000).
https://github.com/randrescastaneda/cache/