TypeError: Cannot read properties of undefined (reading 'arbitrary') for FrequencyArbitrary.js:83:65

[BennieCopeland]

💬 Question and Help

I'm attempting to test that my custom arbitrary types get shrunk to valid types. The following works for records, but I get an exception when using oneof.

import z from "zod";
import fc from "fast-check";
import { describe, it } from "vitest";

export const HalLinkSchema = z.object({
  href: z.string().url("Invalid url"),
  templated: z.boolean().optional(),
  type: z.string().optional(),
  deprecation: z.string().optional(),
  name: z.string().optional(),
  profile: z.string().optional(),
  title: z.string().optional(),
  hreflang: z.string().optional(),
});
type HalLink = z.infer<typeof HalLinkSchema>;

const HalLinksSchema = z.record(
  z.string(),
  z.union([HalLinkSchema, z.array(HalLinkSchema)]),
);
type HalLinks = z.infer<typeof HalLinksSchema>;

interface ResourceObjectSchema {
  _links?: HalLinks;
  _embedded?: EmbeddedSchema;
}

interface EmbeddedSchema {
  [key: string]: ResourceObjectSchema | Array<ResourceObjectSchema>;
}

const EmbeddedSchema: z.ZodType<EmbeddedSchema> = z.lazy(() =>
  z.record(
    z.string(),
    z.union([ResourceObjectSchema, z.array(ResourceObjectSchema)]),
  ),
);
const ResourceObjectSchema = z.object({
  _links: HalLinksSchema.optional(),
  _embedded: EmbeddedSchema.optional(),
});

const LoggedOutApiSchema = ResourceObjectSchema.extend({
  _links: z.object({
    self: HalLinkSchema,
    "tat:login": HalLinkSchema,
  }),
});

const LoggedInApiSchema = ResourceObjectSchema.extend({
  _links: z.object({
    self: HalLinkSchema,
    "tat:logout": HalLinkSchema,
    "tat:user": HalLinkSchema,
  }),
});

const ApiResponseSchema = z.union([LoggedInApiSchema, LoggedOutApiSchema]);

type LoggedOutApi = z.infer<typeof LoggedOutApiSchema>;
type LoggedInApi = z.infer<typeof LoggedInApiSchema>;
type ApiResponse = z.infer<typeof ApiResponseSchema>;

const HrefArbitrary: fc.Arbitrary<string> = fc
  .webUrl({ withFragments: true })
  .map((url) => new URL(url).href)
  .map((url) => url.replace(/'/g, "%27"))
  .noShrink();

const HalLinkArbitrary: fc.Arbitrary<HalLink> = fc.record({
  href: HrefArbitrary.noShrink(),
  templated: fc.option(fc.boolean(), { nil: undefined, freq: 1 }),
  type: fc.option(fc.string(), { nil: undefined, freq: 1 }),
  deprecation: fc.option(fc.string(), { nil: undefined, freq: 1 }),
  name: fc.option(fc.string(), { nil: undefined, freq: 1 }),
  profile: fc.option(fc.string(), { nil: undefined, freq: 1 }),
  title: fc.option(fc.string(), { nil: undefined, freq: 1 }),
  hreflang: fc.option(fc.string(), { nil: undefined, freq: 1 }),
});

const LoggedOutApiArbitrary: fc.Arbitrary<LoggedOutApi> = fc.record({
  _links: fc.record({
    self: HalLinkArbitrary,
    "tat:login": HalLinkArbitrary,
  }),
});

const LoggedInApiArbitrary: fc.Arbitrary<LoggedInApi> = fc.record({
  _links: fc.record({
    self: HalLinkArbitrary,
    "tat:logout": HalLinkArbitrary,
    "tat:user": HalLinkArbitrary,
  }),
});

const ApiResponseArbitrary: fc.Arbitrary<ApiResponse> = fc.oneof(
  { withCrossShrink: false },
  LoggedInApiArbitrary,
  LoggedOutApiArbitrary,
);

it("HalLinkArbitrary.shrink() returns valid objects", () => {
  fc.check(
    fc.property(HalLinkArbitrary, (halLink) => {
      const values = HalLinkArbitrary.shrink(halLink, fc.context());

      return values.every((v) => HalLinkSchema.safeParse(v.value).success);
    }),
  );
});

it("ApiResponseArbitrary generates valid objects", () => {
  fc.assert(
    fc.property(ApiResponseArbitrary, (obj) => {
      ApiResponseSchema.parse(obj);
    }),
  );
});

it("ApiResponseArbitrary.shrink() returns valid objects", () => {
  fc.assert(
    fc.property(ApiResponseArbitrary, fc.context(), (obj, ctx) => {
      // throws an exception here
      const values = ApiResponseArbitrary.shrink(obj, ctx);

      return values.every((v) => ApiResponseSchema.safeParse(v.value).success);
    }),
  );
});

If I call ApiResponseArbitrary.shrink(obj, undefined), it doesn't throw, but it also doesn't return any shrunken values.

To Reproduce

See code above

Expected behavior

Not to throw, and that a list of shrunken values be returned.

Your environment

Packages / Softwares	Version(s)
fast-check	3.15.0
node	21.4.0
TypeScript	5.3.3
zod	3.22.4
vitest	1.0.4

[dubzzz]

Not a bug! You're not using the shrink API the right way.

The shrink API must be provided a context but not a random one. It might be given the context that got built with the value itself. If you call generate on your arbitrary it will provide you a value along with a context. This is the context that should be passed to shrink. Documented at https://fast-check.dev/api-reference/classes/Arbitrary.html#shrink.

In your case, your context is defintely not the one your arbitrary expects. What you can do (as you don't have any context) is to call canShrinkWithoutContext on your arbitrary and if true, you can call shrink with the value and undefined as a context.

But I'm a bit surprised: why do you need to call shrink yourself? If you use it to validate your arbitrary, maybe you should check how I do that on: https://github.com/dubzzz/fast-check/blob/main/packages/fast-check/test/unit/arbitrary/__test-helpers__/ArbitraryAssertions.ts#L77. This helper is the one used to check all arbitraries I provide natively in fast-check.

[BennieCopeland]

I assumed I was using it incorrectly, but I couldn't find anything about how to properly test shrinkers. I also couldn't find any documentation that really explained what the context is doing or how to get ahold of it for the generated value. In F# with FsCheck, shrink doesn't take a context and I can register my own custom one as well as test it. I have a nice little pattern I use to remove a lot of the boilerplate of testing every generator/shrinker combo.

As to why I'm doing it, it's good practice to verify custom generators and shrinks per this video from John Hughes. For example, if I am writing code for a balanced tree, the shrinker should return valid shrunken balanced trees. Same thing for any invariants that can't directly be encoded by the type system, but must be protected via code. I failed to do this at first, and it bit me in the ass. In particular the generated HREFs would get shrunk to "invalid" ones causing vite-fetch-mock to fail. Not a fast-check issue, but an issue with how vite-fetch-mock transforms and matches the URL internally.

It looks like that helper function might be what I'm looking for, as I will still want to test arbitrary's that can only be shrunken with context. I'll see how far I can run with that.

Something else I couldn't find in the docs is how to write/add my own custom shrinker for an Arbitrary. I haven't needed to yet, but it's only a matter of time before I need to.

[dubzzz]

Indeed, the shrinking philosophy used by QuickCheck-like frameworks does not require anything more than the value. But unfortunately that choice makes shrinkers less powerful.

Historically fast-check started with that choice in v0. It got inspired from Jsverify (the leading property based testing option when I started the project). But I moved to a more powerful closure-based approach with v1. In other words in v1, generate method was generating a value and its shrink function.

In v3 (or maybe 2), I introduced a shrink function. This time no more closure needed. The thing that used to be captured via a closure is now captured by a context. It made users able to reuse shrinkers.

[dubzzz]

Regarding "writing your own shrinker", I had a plan to document it but I never found the time to do so.