name 'Serial' is not defined

[hoosnick]

When creating a new migration, I think this bug(error) only occurs with ForeignKeys, details below:

my_app/tables.py

class Order(Table, tablename="orders"):
    user = ForeignKey(
        LazyTableReference(
            table_class_name="Users",
            module_path="users.tables"
        )
    )
    tariff = ForeignKey(
        LazyTableReference(
            table_class_name="Tariff",
            module_path="course.tables"
        )
    )

my_app/piccolo_migrations/my_app_2023_06_30t18_31_03_962663.py

---

Traceback

The command failed.
name 'Serial' is not defined
Traceback (most recent call last):
  File "path\to\project\env\Lib\site-packages\targ\__init__.py", line 448, in run      
    command.call_with(arg_class)
  File "path\to\project\env\Lib\site-packages\targ\__init__.py", line 229, in call_with
    asyncio.run(self.command(**cleaned_kwargs))
  File "path\to\programs\Python\Python311\Lib\asyncio\runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "path\to\programs\Python\Python311\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "path\to\programs\Python\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "path\to\project\env\Lib\site-packages\piccolo\apps\migrations\commands\forwards.py", line 159, in forwards
    response = await run_forwards(
               ^^^^^^^^^^^^^^^^^^^
  File "path\to\project\env\Lib\site-packages\piccolo\apps\migrations\commands\forwards.py", line 120, in run_forwards
    response = await manager.run()
               ^^^^^^^^^^^^^^^^^^^
  File "path\to\project\env\Lib\site-packages\piccolo\apps\migrations\commands\forwards.py", line 97, in run
    return await self.run_migrations(app_config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "path\to\project\env\Lib\site-packages\piccolo\apps\migrations\commands\forwards.py", line 37, in run_migrations
    ] = self.get_migration_modules(app_config.migrations_folder_path)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "path\to\project\env\Lib\site-packages\piccolo\apps\migrations\commands\base.py", line 53, in get_migration_modules
    modules: t.List[MigrationModule] = [
                                       ^
  File "path\to\project\env\Lib\site-packages\piccolo\apps\migrations\commands\base.py", line 54, in <listcomp>
    t.cast(MigrationModule, importlib.import_module(name))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "path\to\programs\Python\Python311\Lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "path\to\project\payment\piccolo_migrations\payment_2023_07_14t19_26_53_265153.py", line 20, in <module>
    class Tariff(Table, tablename="tariff", schema=None):
  File "path\to\project\payment\piccolo_migrations\payment_2023_07_14t19_26_53_265153.py", line 21, in Tariff
    id = Serial(
         ^^^^^^
NameError: name 'Serial' is not defined

[hoosnick]

Serial is not added to extra_imports when rendering the template. Or is it related to --auto?

[dantownsend]

Ah, interesting - is this on the most recent version of Piccolo?

I wonder under what situations it happens? Were you adding a foreign key to an existing table, or was this a new table?

[hoosnick]

I had this problem with one previous version of Piccolo, and now I'm on the latest version.

When I added a foreign key to my forwarded(migrated) User and Tariff tables for Order table.

piccolo migrations new all --auto

...
...
...
✅ Finished

It worked for all apps without problems. But when I use the forwards command to migrate:

piccolo migrations forwards all

                          SESSION_AUTH

----------------------------------------------------------------
👍 2 migrations already complete
🏁 No migrations need to be run

                              USER

----------------------------------------------------------------
👍 4 migrations already complete
🏁 No migrations need to be run

                             COURSE

----------------------------------------------------------------
👍 1 migration already complete
⏩ 1 migration not yet run
🚀 Running 1 migration:
  - 2023-07-14T19:26:53:114807 [forwards]... ok! ✔️

                             USERS

----------------------------------------------------------------
👍 1 migration already complete
⏩ 1 migration not yet run
🚀 Running 1 migration:
  - 2023-07-14T19:26:53:440473 [forwards]... ok! ✔️

                            MY_APP

----------------------------------------------------------------
The command failed.
name 'Serial' is not defined
For a full stack trace, use --trace

[dantownsend]

Yeah, I think I saw this on a previous version of Piccolo. The fix is to manually add the import to the migration file:

from piccolo.columns.column_types import Serial

I need to try and figure out if it's a still a problem (i.e. if you create a new migration does it have this problem, or is it just for migrations created in older versions).

So far I've tried adding a foreign key to an existing table, and that included the Serial import.

[hoosnick]

The fix is to manually add the import to the migration file

Yes, I was doing this solution until now. But I thought this issue was a bug because it appeared in my 2-3 projects where different versions of Piccolo were used.

So far I've tried adding a foreign key to an existing table, and that included the Serial import.

So, do I need to migrate my foreign tables first and then add the foreign key? So sorted apps now can't be a help/solution either?

[hoosnick]

I had an empty db and empty piccolo_migrations folder (per app) in this try.

piccolo migrations new all --auto

piccolo migrations forwards all

[hoosnick]

@dantownsend The problem is with LazyTableReference and was solved when I removed it and imported the model from my other app/tables.py in the tables.py file.

[dantownsend]

@hoosnick Ah, that's good to know.

I was trying to find the commit where I thought I'd fixed the Serial import. This sounds like a new bug then with LazyTableReference.

[dantownsend]

I came across this bug today - you're right, it's caused when creating a migration with a ForeignKey which uses LazyTableReference.

[hoosnick]

@dantownsend Has the bug been fixed?

[dantownsend]

@hoosnick Not yet - I've had a look into it, but haven't created a PR yet.