StreamField migration toolkit cannot handle null stream fields

[jams2]

Issue Summary

If the operations in the StreamField migration toolkit attempt to process a stream field that is null (at the top level), they throw a TypeError:

gitpod /workspace/wagtail-gitpod (main) $ ./manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, home, sessions, taggit, wagtailadmin, wagtailcore, wagtaildocs, wagtailembeds, wagtailforms, wagtailimages, wagtailredirects, wagtailsearch, wagtailusers
Running migrations:
  Applying home.0004_stream_field_migration...Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/workspace/.pip-modules/lib/python3.8/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/workspace/.pip-modules/lib/python3.8/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/workspace/.pip-modules/lib/python3.8/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/workspace/.pip-modules/lib/python3.8/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
  File "/workspace/.pip-modules/lib/python3.8/site-packages/django/core/management/base.py", line 106, in wrapper
    res = handle_func(*args, **kwargs)
  File "/workspace/.pip-modules/lib/python3.8/site-packages/django/core/management/commands/migrate.py", line 356, in handle
    post_migrate_state = executor.migrate(
  File "/workspace/.pip-modules/lib/python3.8/site-packages/django/db/migrations/executor.py", line 135, in migrate
    state = self._migrate_all_forwards(
  File "/workspace/.pip-modules/lib/python3.8/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
    state = self.apply_migration(
  File "/workspace/.pip-modules/lib/python3.8/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/workspace/.pip-modules/lib/python3.8/site-packages/django/db/migrations/migration.py", line 132, in apply
    operation.database_forwards(
  File "/workspace/.pip-modules/lib/python3.8/site-packages/django/db/migrations/operations/special.py", line 193, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/workspace/.pip-modules/lib/python3.8/site-packages/wagtail/blocks/migrations/migrate_operation.py", line 119, in migrate_stream_data_forward
    raw_data = utils.apply_changes_to_raw_data(
  File "/workspace/.pip-modules/lib/python3.8/site-packages/wagtail/blocks/migrations/utils.py", line 283, in apply_changes_to_raw_data
    altered_raw_data = map_block_value(
  File "/workspace/.pip-modules/lib/python3.8/site-packages/wagtail/blocks/migrations/utils.py", line 68, in map_block_value
    return map_stream_block_value(
  File "/workspace/.pip-modules/lib/python3.8/site-packages/wagtail/blocks/migrations/utils.py", line 117, in map_stream_block_value
    for child_block in stream_block_value:
TypeError: 'NoneType' object is not iterable

Originally observed with one of the built-in operations.

Steps to Reproduce

1. Start a new project with wagtail start myproject
2. Edit home/models.py as follows

from django.db import models

from wagtail.models import Page
from wagtail.fields import StreamField
from wagtail import blocks


class HomePage(Page):
    content = StreamField(
        [("text", blocks.TextBlock(required=False))], null=True, blank=True
    )

3. ./manage.py makemigrations
4. ./manage.py migrate
5. ./manage.py makemigrations --empty -n stream_field_migration home
6. edit the new migration as follows:

# Generated by Django 4.2.11 on 2024-04-30 15:20

from django.db import migrations

from wagtail.blocks.migrations.migrate_operation import MigrateStreamData
from wagtail.blocks.migrations.operations import BaseBlockOperation


class NoopOperation(BaseBlockOperation):
    def apply(self, block_value):
        return block_value

    @property
    def operation_name_fragment(self):
        return "noop"


class Migration(migrations.Migration):

    dependencies = [
        ('home', '0003_homepage_content'),
    ]

    operations = [
        MigrateStreamData(
            app_name="home",
            model_name="HomePage",
            field_name="content",
            operations_and_block_paths=[
                (NoopOperation(), "text"),
            ]
        )
    ]

7. ./manage.py migrate

I think the operations should handle null stream field data gracefully, by not attempting to process such data.

• I have confirmed that this issue can be reproduced as described on a fresh Wagtail project: yes

Technical details

• Python version: Python 3.8.12
• Django version: 4.2.11
• Wagtail version: 6.0.2
• Browser version: N/A

Working on this

I am happy to submit a patch for this issue.

[PARTHIB-DEB]

Hey @jams2 , This is Parthib From India. I have cloned this repo , created a project by wagtail , done the following steps and this error came like you described. I think there is some issue regarding the fields of the model because it is saying that TypeError: 'NoneType' object is not iterable. If you assign me this problem then , I may look for the actual place where the loophole of the logic is resided.