This Development-cycle in Cargo: 1.83

Oct. 31, 2024 · Ed Page on behalf of The Cargo Team

This Development-cycle in Cargo: 1.83

This is a summary of what has been happening around Cargo development for the last 6 weeks which is approximately the merge window for Rust 1.83.

Plugin of the cycle

Cargo can't be everything to everyone, if for no other reason than the compatibility guarantees it must uphold. Plugins play an important part of the Cargo ecosystem and we want to celebrate them.

Our plugin for this cycle is cargo-bloat which shows what is taking up space in your binary.

Thanks to epage for the suggestion!

Please submit your suggestions for the next post.

Implementation

MSRV-aware Cargo

Update from 1.82

In preparation for documenting the MSRV-aware dependency resolver, epage revamped the dependency resolution documentation ( #14620, #14662, ) and wrote down the lessons learned on choosing an MSRV from the RFC process (#14636).

Previous, we discussed suboptimal results when resolving dependencies with multiple distinct MSRVs in a workspace. We were resolving for the lowest MSRV in the workspace. For a workspace member with any other MSRV, they would either get a version older than needed or too new. We proposed resolving for compatibility with the most MSRVs in the workspace. We made reporting improvements to better communicate this. However, there was still concerns over newer users unintentionally ending up in this situation and being confused. With the MSRV policy recommendations we made in #14636, we are directing users away from this situation. In comparing with the current state and the remaining problems with the proposed state, we felt the balance shifted in favor of the proposed state and we merged it in #14569.

At this point, epage posted #14639 to stabilize and document the MSRV-aware resolver. FCP did not close before the end of 1.83, so expect this for 1.84.

We also discussed and further refined the dependency change output in 14568.

Shell completions

Update from 1.82

With clap's new completion system integrated into nightly Cargo in #14493, shannmu added custom completers in an effort to get parity with the hand-maintained completion scripts, including:

In testing this out, epage

  • Fixed a bug with bash (#5731)
  • Made this work with cargo +nightly (clap#5733) and subsequently fixed --target completion for rustup which that broke (#14564)
  • Made completions err on the side of giving no completions rather than completing file names (clap#5763)
  • Tweaked the display order (clap#5739, clap#5741, clap#5743)
  • Removed redundant completions (clap#5740)

One issue that came up is that cargo check --bin <TAB> shows both binary names as well as all check options. Options like --bin are defined to optionally take a value (i.e. --bin [NAME] or num_args(0..=1)) so that a user without completions can run cargo check --bin<ENTER> and see the list of names they can choose from. Since the new completer is working off of the definitions of the CLI, it completes --bin both for taking a value and not, not knowing that the value is required in practice and it should only complete values. The ideal solution would be to delegate the "possible values" reporting to clap so users get both the nicer error reporting and high quality completions. We are tracking this in #14606.

In #14592, epage made it easier to profile custom completers as Cargo's overhead is important for the usability of this feature.

For how to try this feature out, see the documentation.

Public/private dependencies

Update from 1.77

While some improvements have been made to public/private dependency support ( #14504, #14507 ), this feature has been blocked on bugs in the Rustc lint ( rust#71043, rust#119428, ). This is blocking other improvements that can build off of knowing which dependencies are private or not, including:

  • cargo-semver-checks being able to identify when a public dependency's major version is bumped which becomes important for inheriting workspace dependencies because action-at-a-distance can be a breaking change
  • Speeding up cargo doc by skipping unreachable transitive dependencies (#2025)
  • An alternative to cargo doc --no-deps that will generate documentation for all packages relevant to the API
  • Smarter heuristics for cargo add in picking a version requirements for dependencies
  • A way to keep version requirements in-sync

On zulip, epage floated the idea of stabilizing the support in Cargo without the Rustc lints. Alternatively, we could limit the scope of the Rustc lints to err on the side of false negatives over false positives. This needs more planning and coordination with the compiler team.

Optimizing cargo

Update from 1.81

On zulip, some tool authors were discussing with the Cargo team how to optimize cargo metadata.

One of the concerns raised was with the serialization / deserialization time of json. In considering supporting another format, there has to be enough of a difference from json to be worth the effort, it must have a good compatibility story, and we should consider the interoperability of the format. For example, CBOR was brought up. While it has an IETF RFC, the Rust implementations have had some incompatibilities between them and the deserialization time seems to be on par with json (source) though real world benchmarks with cargo metadata would be needed to tell if it will make a difference.

In other performance news, x-hgg-x has been optimizing the dependency resolver ( #14663, #14665, #14692 ). They also proposed we change the allocator used in Cargo (#14691). This is being discussed on zulip.

Design discussions

Target and target

Update from 1.82

We previously discussed wanting a --all-targets-and-doctests flag. While that name is good enough for development, we might not want it when stabilized. If we had an alternative name for "targets" we could call this --all-<something> and deprecate --all-targets. This isn't the first time renaming "cargo target" or "build target" has come up.

In Cargo, the word "target" can refer to

  • An independent build artifact
  • The definition and source for a build artifact (see Cargo targets)
  • build.target-dir where build artifacts and their build state are stored, organized by platform triple when cross-compiling
  • build.target which is the platform triple being built for
  • Manifest [target] and config [target] tables which are ostensibly for a platform triple but allow almost any cfg expression

On the CLI, this looks like:

  • --all-targets
  • --target-dir <PATH>
  • --target <TRIPLE>

The language gets muddier because a "build target" is a top-level build artifact for a given Cargo.toml file while a "build unit" or a "crate" includes top-level build artifacts, associated build artifacts (e.g. build scripts, doctests, and unit tests), any of their dependencies needed for building. To make this even more confusing, while Cargo and Rustc use that definition of "crate", the more commonly known definition is that associated with a [package] as used by cargo publish and crates.io.

While some users may carefully check the documentation or already be very familiar with these corners of Cargo, we should still consider the impact on users who are new to Cargo or not had a reason to dive into these corners yet. Just talking about these topics, it takes care to disambiguate how the terms are being used.

Focusing specifically on disambiguating "target", build.target and [target] (roughly platform triple) are heavily ingrained in Cargo and the community and would have a high cost to try to transition away from. In 1.82, we talked about replacing build.target-dir (#14125) and --all-targets (#6669) for other reasons. That only leaves documentation for where "build target" is used (actually "Cargo target"), making it the easiest use of the word "target" to change.

In #14125, we were looking at making one of the replacements for build.target-dir be build.artifacts-dir. Similarly, when overriding which "build target" is accessed within a dependency's Cargo.toml, we were calling that an artifact dependency. Our natural inclination is to then use the term "artifact" for "build target". However, this showed a slight disconnect between how each of these terms is used:

  • Cargo only copies into the build.artifacts-dir binaries, dynamic libraries, and explicitly-requested-on-the-command-line static libraries and not all "build targets" though there is a request for such behavior
  • A single "build target" can have multiple output files, e.g. debug files. We can likely gloss over this.
  • "build target" can also refer to the definition and source though this is a bit of a misnomer built from using "build target" as a metonym for the definition

Other names that have come up include

  • component
  • product
  • output

The conversation for this is continuing on Internals

Linting

Update from 1.80

Urgau is looking at extending the check-cfg linting system in rustc to Cargo to apply to target.<cfg> tables (#14581). While there are several new-to-Cargo aspects to work out (or defer), one in particular we discussed was what namespace the lint level and configuration should live under.

The current proposal is to reuse the Rustc lint in Cargo, e.g.

[lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(foo)'] }

However, this potentially applies to other Rust-like lints and groups we may want in Cargo, including

By putting these in lints.cargo, users will need to duplicate their unexpected_cfgs configuration. However, other lints like deprecated or non_snake_case they may want to control separately between Cargo and Rust.

By putting these in lints.rust,

  • We need to figure out what is responsible for linting for unknown_lints that both knows about both Cargo and Rust rust:: lints and ensures no more than one message is sent per instance
  • We need to figure out a process for Cargo extending the namespace with new lints (e.g. any buy-in from T-compiler about adding rust::non_kebab_case)
  • Cargo will need a way to calculate effective lint levels after processing of lint groups
  • We need to decide if RUSTFLAGS can also affect Cargo's linting as users are used to it applying to the rust namespace

This isn't just for the rust namespace. There is still an open question on whether Cargo should have an explicit lint command that fills a role like clippy and if that should be a part of cargo clippy somehow. If so, then Cargo will also have lints in the clippy namespace.

This is being discussed on zulip.

Build Script API

The Cargo team has been discussing making more official APIs for interacting with Cargo, for instance an API for build script authors (#12432).

In these discussions, we had overlooked RFC 3119 which established policies around packages owned by the Rust Project. In #14599, we used our team charter to override the default process for adding new packages. We then used that new process in 14600 to officially declare the support policy for each package in the Cargo repo.

With that out of the way, we used the process in #12432 to officially accept a build script API package.

Misc

  • Daily reports by Eh2406 on the progress of the Rust implementation of the PugGrub version solving algorithm
  • Flowrey finished their work on a --dry-run flag for cargo install (#14280)
  • ahaoboy added dark mode to --timings (#14588)
  • Xaeroxe added unstable support using checksums, instead of mtimes, to determine when to rebuild (#14137)
  • epage posted RFC 3692 which came about from several conversations people had with the Cargo team at RustConf
  • In response to #14555, the Cargo team decided to put the responsibility for mixing programmatic and human output on wrapper tools for now

Focus areas without progress

These are areas of interest for Cargo team members with no reportable progress for this development-cycle.

Ready-to-develop:

Needs design and/or experimentation:

Planning:

How you can help

If you have ideas for improving cargo, we recommend first checking our backlog and then exploring the idea on Internals.

If there is a particular issue that you are wanting resolved that wasn't discussed here, some steps you can take to help move it along include:

  • Summarizing the existing conversation (example: Better support for docker layer caching, Change in Cargo.lock policy, MSRV-aware resolver )
  • Document prior art from other ecosystems so we can build on the work others have done and make something familiar to users, where it makes sense
  • Document related problems and solutions within Cargo so we see if we are solving to the right layer of abstraction
  • Building on those posts, propose a solution that takes into account the above information and cargo's compatibility requirements (example)

We are available to help mentor people for S-accepted issues on zulip and you can talk to us in real-time during Contributor Office Hours. If you are looking to help with one of the bigger projects mentioned here and are just starting out, fixing some issues will help familiarize yourself with the process and expectations, making things go more smoothly. If you'd like to tackle something without a mentor, the expectations will be higher on what you'll need to do on your own.