A new nightly version has been released!
See the quickstart guide for installation instructions: Quickstart | Modular
MAX changelog updates:
- Changes to
max generate
:- Added
--top-k
,--temperature
, and--seed
flags. - Changed
--num-warmups
behavior. Previously, it would run the model
on the promptN
times, generating until reaching a stop condition each
time. Now,--num-warmups
runs the model forN
steps instead, generating
N
new tokens as a warmup.
- Added
- Added
generate()
toTextGenerationPipeline
and
SpeculativeDecodingPipeline
, a convenience method for getting text
generations.generate_async
is available for getting streamed outputs.
Mojo changelog updates:
Raw MAX diff: https://github.com/modular/modular/compare/2ca4e563565cc24dc0891f78a1f159781b7ea150...e3fc382cd80729bc0ffe641a67bdd125d1ffce8e)>
Current Mojo changelog: https://github.com/modular/modular/blob/main/mojo/docs/changelog.md