v0.4.0

Major release: v0.4.0 is here! 🚀

This release adds several new features and major refactorings in both modeling and environment sides!

Changelog

✨ Features

• DeepACO + ACO @Furffico @henry-yeh
• Non-autoregressive (NAR) models and NARGNN @Furffico @henry-yeh
• Add modular environment data generator with support to new distributions @cbhua
• New decoding techniques based on the decoding strategy class @LTluttmann
  • Top-p (nucleus sampling)
  • Top-k
• Select start nodes functions @LTluttmann

⚙️ Refactoring

• Major modeling refactoring (summarized here). Now we categorize NCO approaches (which are not necessarily trained with RL!) into the following: 1) constructive (AR and NAR), 2) improvement, 3) transductive. This translates into code, which is now fully customizable. For instance, in constructive methods, now encoders / decoders can be fully replaced or removed in an abstract way!
• Major environment refactoring (summarized here): we further modularize the environments into components (logic under env, data generation under generator, and so on), with several components moved inside the RL4COEnvBase. Importantly, we introduce data generators that can be customized!
• Use Abstract classes if class should not be @ngastzepeda

📝 Documentation

• Hydra documentation and tutorial @LTluttmann
• New modularized examples under examples/
• Updated RL4CO structure in ReadTheDocs
• Move to MIT license with AI4CO for inclusiveness
• New RL4CO / AI4CO swag. You may also find them here!

🐛 Bug Fixes

• MatNet and FFSP bugfix @LTluttmann
• Best solution gathering from POMO @ahottung
• Tests now passing on MPS; compatibility with TorchRL pytorch/rl#2125
• Miscellaneuous @LTluttmann , @bokveizen , @tycbony

v0.3.3

New Routing Envs and more 🚀

Changelog

✨ Features

• Add CVRPTW Environment @ngastzepeda
  • Add Solomon instance / solution loader via vrplib
• Add basic Skill-VRP (SVRP) @ngastzepeda

📃 Documentation

• [Minor] improve decoding strategies documentation

🐛 Bug Fixes

• Avoid deepcopy bug by not saving intermediate steps of decoding strategy #123
• Allow passing select_start_nodes_fn and other kwargs in decoding strategies

v0.3.2

New Decoding Types and more 🚀

Changelog

Features

• Beam Search #109 #110 @LTluttmann
• Decoding type class #109 #110 @LTluttmann

Documentation

• Add (simple , API work in progress!) tutorial notebooks for TSPLib and CVRPLib #84
• Add decoding strategies notebook @LTluttmann + small fix @Haimrich

Optimization

• torch.no_grad to torch.inference_mode
• Faster testing

Bug Fixes

• Batch size initialization @ngastzepeda
• Bump up naming to align with 0.4.0 release of TorchRL
• MatNet bug fix #108

v0.3.1

QoL and BugFixes 🚀

Changelog

• Better multi start decoding #102
  • Add modular select_start_nodes function for POMO
  • Improve efficiency of multistart function
  • Add testing and selection function for more envs
  • Fix OP selecting too far away nodes in POMO
  • Automatic multistart, no need to manually choose beforehand when running POMO
• Fix CVRP capacity bug @ngastzepeda #105
• Add critic init embedding support
• Fix data generation and add better docs #106
• Better dataset handling: add dataset choice; use low CPU usage dataset by default
• Better solution plotting and better quickstart notebook #103
• Library winter cleanup
• Miscellaneous minor fixes here and there

v0.3.0

Faster Library, Python 3.11 and new TorchRL support, Envs, Models, Multiple Dataloaders, and more 🚀

Faster Library, new Python 3.11 and TorchRL

• Update to latest TorchRL #72, solving several issues as #95 #97 (also see this)
• Benchmarking:
  • Up to 20% speedup in training epochs thanks to faster TensorDict and new env updates
  • Almost instant data generation (avoid list comprehension, e.g. from ~20 seconds to <1 second per epoch!)
• Python 3.11 now available #97

New SMTWTP environment

• Add new scheduling problem: Single Machine Total Weighted Tardiness Problem environment as in DeepACO @henry-yeh

New MatNet model

• Add MatNet version for square matrices (faster implementation ideal for routing problems)
• Should be easy to implement scheduling from here

Multiple Dataloaders

• Now it is possible to have multiple dataloaders, with naming as well!
  • For example, to track generalization during training

Miscellaneous

• Fix POMO shapes @hyeok9855 , modularizing PPO etc
• Fix precion bug for PPO
• New AI4CO transfer!

v0.2.3

Add FlashAttention2 support ⚡

• Add FlashAttention2 support as mentioned here
• Remove old wrapper for half() precision since Lightning already deals with this
• Fix scaled_dot_product_attention implementation in PyTorch < 2.0
• Minor fixes

v0.2.2

QoL: New Baseline, Testing Search Methods, Downloader, Miscellanea 🚀

Changelog

• Add mean baseline @hyeok9855
• Add testing for search methods
• Move downloader to external repo, extra URL as backup for DPP
• Small bug fix for duplicate args
• Add more modular data generation
• Suppress extra warning in automatic_optimization
• Minor doc cleaning

v0.2.1

QoL, Better documentation, Bug Fixes 🚀

• Add RandomPolicy class
• Control max_steps for debugging purposes during decoding
• Better documentation, add tutorials, and references #88 @bokveizen
• Set bound to < Python 3.11 for the time being #90 @hyeok9855
• Log more info by default in PPO
• precompute_cache method can now accept td as well
• If Trainer is supplied with gradient_clip_val and manual_optimization=False, then remove gradient clipping (e.g. for PPO)
• Fix test data size following training and not test by default

v0.2.0

Search Methods, Flexible Embeddings, New Graph Encoders and more 🚀

Search methods

• New flexible and extensible abstract class
• Active Search (Bello et al, 2016)
• Efficient Active Search (Hottung et al, 2022)

Flexible embeddings

• Support for changing any environment embedding (init, context and dynamic)
• Add new notebook showcasing how to solve new complex problems (example of multi-depot multi-agent pickup and delivery problem - MDPDP)

Support for torch-geometric

• Added new template graph neural networks (MPNN, GCN)
• Example Notebook here

Miscellaneous

• Separate loggers
• Better imports
• Bugfix compatibility with Mac
• Update configs
• ... and more!

v0.1.1

Better training, Bug fixes, and more 🚀

• Better automatic training with DDP #87
• Bug Fix RL4COTrainer
• Avoid broadcasting error warning in critic baselines
• Fix rollout baseline bug
• New experiment config structure: interpolate with environment name (we won't need anymore to have separate folders for each environment name such as TSP, CVRP etc, simply use one config to rule them all!