Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crd properties with - in name #43

Open
henderiw opened this issue May 15, 2021 · 12 comments · Fixed by pulumi/pulumi#10738, pulumi/pulumi#11049 or #99
Open

crd properties with - in name #43

henderiw opened this issue May 15, 2021 · 12 comments · Fixed by pulumi/pulumi#10738, pulumi/pulumi#11049 or #99
Labels
area/codegen Affects quality or correctness of generated code kind/bug Some behavior is incorrect or out of spec
Milestone

Comments

@henderiw
Copy link

henderiw commented May 15, 2021

I am using a crd which has property names with - in it and the crd2pulumi failed with

panic: invalid Go source code:

crds/srlinux/v1alpha1/pulumiTypes.go
: 269:7: expected ';', found '-' (and 10 more errors)

goroutine 1 [running]:
github.com/pulumi/pulumi/pkg/v3/codegen/go.GeneratePackage.func1(0xc0001198f0, 0x24, 0xc000a88000, 0x497e4)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/pkg/v3@v3.0.0/codegen/go/gen.go:2151 +0x46e
github.com/pulumi/pulumi/pkg/v3/codegen/go.GeneratePackage(0x1df1de5, 0xa, 0xc000532280, 0x2, 0xc000026508, 0x1)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/pkg/v3@v3.0.0/codegen/go/gen.go:2272 +0x1b33
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genGoFiles(0xc0002a5a90, 0x1de94c9, 0x4, 0xc0002163b0, 0x1, 0x1)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/golang.go:55 +0x345
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genGo(0xc0002a5a90, 0xc00039b4a8, 0x7, 0x1de94c9, 0x4, 0x1, 0xc0002163f0)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/golang.go:32 +0x45
github.com/pulumi/crd2pulumi/gen.Generate(0x0, 0x0, 0x0, 0xc00011a7d0, 0x1de94c9, 0x4, 0x1de94c9, 0x4, 0x1de94c9, 0x4, ...)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:73 +0x20c
github.com/pulumi/crd2pulumi/cmd.Execute.func2(0xc00047d080, 0xc000716240, 0x1, 0x3)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:157 +0x271
github.com/spf13/cobra.(*Command).execute(0xc00047d080, 0xc00003a090, 0x3, 0x3, 0xc00047d080, 0xc00003a090)
	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:846 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0xc00047d080, 0xc00013ff00, 0x1, 0x1)
	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887
github.com/pulumi/crd2pulumi/cmd.Execute(0xc00007a748, 0x263f000)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:189 +0x666
main.main()

input file:

          spec:
            description: SrlInterfaceSpec struct
            properties:
              interface:
                items:
                  description: Interface struct
                  properties:
                    admin-state:
                      default: enable
                      enum:
                      - disable
                      - enable
                      type: string

I changed the code as per below and this solves it for me.

// AddType converts the given OpenAPI `schema` to a ObjectTypeSpec and adds it
// to the `types` map under the given `name`. Recursively converts and adds all
// nested schemas as well.
func AddType(schema map[string]interface{}, name string, types map[string]pschema.ComplexTypeSpec) {
	properties, foundProperties, _ := unstruct.NestedMap(schema, "properties")
	description, _, _ := unstruct.NestedString(schema, "description")
	schemaType, _, _ := unstruct.NestedString(schema, "type")
	required, _, _ := unstruct.NestedStringSlice(schema, "required")

	propertySpecs := map[string]pschema.PropertySpec{}
	for propertyName := range properties {
		propertyName = strcase.UpperCamelCase(propertyName)
		propertySchema, _, _ := unstruct.NestedMap(properties, propertyName)
		propertyDescription, _, _ := unstruct.NestedString(propertySchema, "description")
		defaultValue, _, _ := unstruct.NestedFieldNoCopy(propertySchema, "default")
		propertySpecs[propertyName] = pschema.PropertySpec{
			TypeSpec:    GetTypeSpec(propertySchema, name+strings.Title(propertyName), types),
			Description: propertyDescription,
			Default:     defaultValue,
		}
	}
@henderiw henderiw added the kind/bug Some behavior is incorrect or out of spec label May 15, 2021
@lblackstone lblackstone added kind/bug Some behavior is incorrect or out of spec and removed kind/bug Some behavior is incorrect or out of spec labels May 18, 2021
@joeduffy
Copy link
Member

Unless I'm mistaken, this is likely more complicated than "just" fixing the crd2pulumi tool (I tried).

First, we would need to translate both type and property names so that the Pulumi names don't contain . and - characters, both of which are valid for CRDs.

However, unless I'm mistaken, we don't currently support having a different Pulumi names for the CRD's type or property names, since we use type tokens and direct names in most places, and as far as I can tell, rely on the Pulumi resource object's names matching at runtime too. (An alternative approach would be to have crd2pulumi generate types that manually marshal a map of CRD names to their Pulumi equivalents, the latter of which would be cleaned of . and -s.)

@henderiw, are you ok sharing your repro file here?

@henderiw
Copy link
Author

henderiw commented May 23, 2021

Yes happy to share the CRD file I am using which triggered [this](url)

---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.4.1
  creationTimestamp: null
  name: srlinterfaces.srlinux.henderiw.be
spec:
  group: srlinux.henderiw.be
  names:
    kind: SrlInterface
    listKind: SrlInterfaceList
    plural: srlinterfaces
    singular: srlinterface
  scope: Namespaced
  versions:
  - name: v1alpha1
    schema:
      openAPIV3Schema:
        description: SrlInterface is the Schema for the SrlInterfaces API
        properties:
          apiVersion:
            description: 'APIVersion defines the versioned schema of this representation
              of an object. Servers should convert recognized schemas to the latest
              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
            type: string
          kind:
            description: 'Kind is a string value representing the REST resource this
              object represents. Servers may infer this from the endpoint the client
              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
            type: string
          metadata:
            type: object
          spec:
            description: SrlInterfaceSpec struct
            properties:
              interface:
                items:
                  description: Interface struct
                  properties:
                    admin-state:
                      default: enable
                      enum:
                      - disable
                      - enable
                      type: string
                    description:
                      maxLength: 255
                      minLength: 1
                      pattern: '[A-Za-z0-9 !@#$^&()|+=`~.,''/_:;?-]*'
                      type: string
                    ethernet:
                      description: InterfaceEthernet struct
                      properties:
                        aggregate-id:
                          type: string
                        auto-negotiate:
                          type: boolean
                        duplex-mode:
                          enum:
                          - full
                          - half
                          type: string
                        flow-control:
                          description: InterfaceEthernetFlowControl struct
                          properties:
                            receive:
                              type: boolean
                            transmit:
                              type: boolean
                          type: object
                        lacp-port-priority:
                          maximum: 65535
                          minimum: 0
                          type: integer
                        port-speed:
                          enum:
                          - 100G
                          - 100M
                          - 10G
                          - 10M
                          - 1G
                          - 1T
                          - 200G
                          - 25G
                          - 400G
                          - 40G
                          - 50G
                          type: string
                      type: object
                    lag:
                      description: InterfaceLag struct
                      properties:
                        lacp:
                          description: InterfaceLagLacp struct
                          properties:
                            admin-key:
                              maximum: 65535
                              minimum: 1
                              type: integer
                            interval:
                              default: SLOW
                              enum:
                              - FAST
                              - SLOW
                              type: string
                            lacp-mode:
                              default: ACTIVE
                              enum:
                              - ACTIVE
                              - PASSIVE
                              type: string
                            system-id-mac:
                              pattern: '[0-9a-fA-F]{2}(:[0-9a-fA-F]{2}){5}'
                              type: string
                            system-priority:
                              maximum: 65535
                              minimum: 0
                              type: integer
                          type: object
                        lacp-fallback-mode:
                          enum:
                          - static
                          type: string
                        lacp-fallback-timeout:
                          maximum: 3600
                          minimum: 4
                          type: integer
                        lag-type:
                          default: static
                          enum:
                          - lacp
                          - static
                          type: string
                        member-speed:
                          enum:
                          - 100G
                          - 100M
                          - 10G
                          - 10M
                          - 1G
                          - 25G
                          - 400G
                          - 40G
                          type: string
                        min-links:
                          default: 1
                          maximum: 64
                          minimum: 1
                          type: integer
                      type: object
                    loopback-mode:
                      type: boolean
                    mtu:
                      maximum: 9500
                      minimum: 1500
                      type: integer
                    name:
                      maxLength: 20
                      minLength: 3
                      pattern: (mgmt0|mgmt0-standby|system0|lo(0|1[0-9][0-9]|2([0-4][0-9]|5[0-5])|[1-9][0-9]|[1-9])|ethernet-([1-9](\d){0,1}(/[abcd])?(/[1-9](\d){0,1})?/(([1-9](\d){0,1})|(1[0-1]\d)|(12[0-8])))|irb(0|1[0-9][0-9]|2([0-4][0-9]|5[0-5])|[1-9][0-9]|[1-9])|lag(([1-9](\d){0,1})|(1[0-1]\d)|(12[0-8])))
                      type: string
                    qos:
                      description: InterfaceQos struct
                      properties:
                        output:
                          description: InterfaceQosOutput struct
                          properties:
                            multicast-queue:
                              items:
                                description: InterfaceQosOutputMulticastQueue struct
                                properties:
                                  queue-id:
                                    maximum: 7
                                    minimum: 0
                                    type: integer
                                  scheduling:
                                    description: InterfaceQosOutputMulticastQueueScheduling
                                      struct
                                    properties:
                                      peak-rate-percent:
                                        default: 100
                                        maximum: 100
                                        minimum: 1
                                        type: integer
                                    type: object
                                  template:
                                    type: string
                                required:
                                - queue-id
                                type: object
                              type: array
                            scheduler:
                              description: InterfaceQosOutputScheduler struct
                              properties:
                                tier:
                                  items:
                                    description: InterfaceQosOutputSchedulerTier struct
                                    properties:
                                      level:
                                        maximum: 4
                                        minimum: 1
                                        type: integer
                                      node:
                                        items:
                                          description: InterfaceQosOutputSchedulerTierNode
                                            struct
                                          properties:
                                            node-number:
                                              maximum: 11
                                              minimum: 0
                                              type: integer
                                            strict-priority:
                                              type: boolean
                                            weight:
                                              default: 1
                                              maximum: 127
                                              minimum: 1
                                              type: integer
                                          required:
                                          - node-number
                                          type: object
                                        type: array
                                    required:
                                    - level
                                    type: object
                                  type: array
                              type: object
                            unicast-queue:
                              items:
                                description: InterfaceQosOutputUnicastQueue struct
                                properties:
                                  queue-id:
                                    maximum: 7
                                    minimum: 0
                                    type: integer
                                  scheduling:
                                    description: InterfaceQosOutputUnicastQueueScheduling
                                      struct
                                    properties:
                                      peak-rate-percent:
                                        default: 100
                                        maximum: 100
                                        minimum: 1
                                        type: integer
                                      strict-priority:
                                        default: true
                                        type: boolean
                                      weight:
                                        default: 1
                                        maximum: 255
                                        minimum: 1
                                        type: integer
                                    type: object
                                  template:
                                    type: string
                                  voq-template:
                                    type: string
                                required:
                                - queue-id
                                type: object
                              type: array
                          type: object
                      type: object
                    sflow:
                      description: InterfaceSflow struct
                      properties:
                        admin-state:
                          enum:
                          - disable
                          - enable
                          type: string
                      type: object
                    transceiver:
                      description: InterfaceTransceiver struct
                      properties:
                        ddm-events:
                          type: boolean
                        forward-error-correction:
                          default: disabled
                          enum:
                          - base-r
                          - disabled
                          - rs-108
                          - rs-528
                          - rs-544
                          type: string
                        tx-laser:
                          type: boolean
                      type: object
                    vlan-tagging:
                      type: boolean
                  required:
                  - name
                  type: object
                type: array
            required:
            - interface
            type: object
          status:
            description: SrlInterfaceStatus struct
            properties:
              configurationDependencyTargetFound:
                description: ConfigurationDependencyTargetNotFound identifies if the
                  target of the resource object is missing or not
                enum:
                - Success
                - Failed
                type: string
              configurationDependencyValidationStatus:
                description: ConfigurationDependencyValidationStatus identifies the
                  status of the LeafRef Validation of the resource object
                enum:
                - Success
                - Failed
                type: string
              lastUpdated:
                description: LastUpdated identifies when this status was last observed.
                format: date-time
                type: string
              targetStatus:
                additionalProperties:
                  description: TargetStatus provides the status of the configuration
                    applied on this particular device
                  properties:
                    configStatus:
                      default: ""
                      description: ConfigStatus defines the states the resource object
                        is reporting
                      enum:
                      - ""
                      - Deleting
                      - DeleteFailed
                      - DeleteSuccess
                      - Configuring
                      - ConfiguredSuccess
                      - ConfigStatusConfigureFailed
                      type: string
                    configStatusDetails:
                      default: ""
                      type: string
                    errorCount:
                      default: 0
                      description: ErrorCount records how many times the host has
                        encoutered an error since the last successful operation
                      type: integer
                  required:
                  - configStatus
                  - errorCount
                  type: object
                description: Target provides the status of the configuration on the
                  device
                type: object
              usedSpec:
                description: UsedSpec provides the spec used for the configuration
                properties:
                  interface:
                    items:
                      description: Interface struct
                      properties:
                        admin-state:
                          default: enable
                          enum:
                          - disable
                          - enable
                          type: string
                        description:
                          maxLength: 255
                          minLength: 1
                          pattern: '[A-Za-z0-9 !@#$^&()|+=`~.,''/_:;?-]*'
                          type: string
                        ethernet:
                          description: InterfaceEthernet struct
                          properties:
                            aggregate-id:
                              type: string
                            auto-negotiate:
                              type: boolean
                            duplex-mode:
                              enum:
                              - full
                              - half
                              type: string
                            flow-control:
                              description: InterfaceEthernetFlowControl struct
                              properties:
                                receive:
                                  type: boolean
                                transmit:
                                  type: boolean
                              type: object
                            lacp-port-priority:
                              maximum: 65535
                              minimum: 0
                              type: integer
                            port-speed:
                              enum:
                              - 100G
                              - 100M
                              - 10G
                              - 10M
                              - 1G
                              - 1T
                              - 200G
                              - 25G
                              - 400G
                              - 40G
                              - 50G
                              type: string
                          type: object
                        lag:
                          description: InterfaceLag struct
                          properties:
                            lacp:
                              description: InterfaceLagLacp struct
                              properties:
                                admin-key:
                                  maximum: 65535
                                  minimum: 1
                                  type: integer
                                interval:
                                  default: SLOW
                                  enum:
                                  - FAST
                                  - SLOW
                                  type: string
                                lacp-mode:
                                  default: ACTIVE
                                  enum:
                                  - ACTIVE
                                  - PASSIVE
                                  type: string
                                system-id-mac:
                                  pattern: '[0-9a-fA-F]{2}(:[0-9a-fA-F]{2}){5}'
                                  type: string
                                system-priority:
                                  maximum: 65535
                                  minimum: 0
                                  type: integer
                              type: object
                            lacp-fallback-mode:
                              enum:
                              - static
                              type: string
                            lacp-fallback-timeout:
                              maximum: 3600
                              minimum: 4
                              type: integer
                            lag-type:
                              default: static
                              enum:
                              - lacp
                              - static
                              type: string
                            member-speed:
                              enum:
                              - 100G
                              - 100M
                              - 10G
                              - 10M
                              - 1G
                              - 25G
                              - 400G
                              - 40G
                              type: string
                            min-links:
                              default: 1
                              maximum: 64
                              minimum: 1
                              type: integer
                          type: object
                        loopback-mode:
                          type: boolean
                        mtu:
                          maximum: 9500
                          minimum: 1500
                          type: integer
                        name:
                          maxLength: 20
                          minLength: 3
                          pattern: (mgmt0|mgmt0-standby|system0|lo(0|1[0-9][0-9]|2([0-4][0-9]|5[0-5])|[1-9][0-9]|[1-9])|ethernet-([1-9](\d){0,1}(/[abcd])?(/[1-9](\d){0,1})?/(([1-9](\d){0,1})|(1[0-1]\d)|(12[0-8])))|irb(0|1[0-9][0-9]|2([0-4][0-9]|5[0-5])|[1-9][0-9]|[1-9])|lag(([1-9](\d){0,1})|(1[0-1]\d)|(12[0-8])))
                          type: string
                        qos:
                          description: InterfaceQos struct
                          properties:
                            output:
                              description: InterfaceQosOutput struct
                              properties:
                                multicast-queue:
                                  items:
                                    description: InterfaceQosOutputMulticastQueue
                                      struct
                                    properties:
                                      queue-id:
                                        maximum: 7
                                        minimum: 0
                                        type: integer
                                      scheduling:
                                        description: InterfaceQosOutputMulticastQueueScheduling
                                          struct
                                        properties:
                                          peak-rate-percent:
                                            default: 100
                                            maximum: 100
                                            minimum: 1
                                            type: integer
                                        type: object
                                      template:
                                        type: string
                                    required:
                                    - queue-id
                                    type: object
                                  type: array
                                scheduler:
                                  description: InterfaceQosOutputScheduler struct
                                  properties:
                                    tier:
                                      items:
                                        description: InterfaceQosOutputSchedulerTier
                                          struct
                                        properties:
                                          level:
                                            maximum: 4
                                            minimum: 1
                                            type: integer
                                          node:
                                            items:
                                              description: InterfaceQosOutputSchedulerTierNode
                                                struct
                                              properties:
                                                node-number:
                                                  maximum: 11
                                                  minimum: 0
                                                  type: integer
                                                strict-priority:
                                                  type: boolean
                                                weight:
                                                  default: 1
                                                  maximum: 127
                                                  minimum: 1
                                                  type: integer
                                              required:
                                              - node-number
                                              type: object
                                            type: array
                                        required:
                                        - level
                                        type: object
                                      type: array
                                  type: object
                                unicast-queue:
                                  items:
                                    description: InterfaceQosOutputUnicastQueue struct
                                    properties:
                                      queue-id:
                                        maximum: 7
                                        minimum: 0
                                        type: integer
                                      scheduling:
                                        description: InterfaceQosOutputUnicastQueueScheduling
                                          struct
                                        properties:
                                          peak-rate-percent:
                                            default: 100
                                            maximum: 100
                                            minimum: 1
                                            type: integer
                                          strict-priority:
                                            default: true
                                            type: boolean
                                          weight:
                                            default: 1
                                            maximum: 255
                                            minimum: 1
                                            type: integer
                                        type: object
                                      template:
                                        type: string
                                      voq-template:
                                        type: string
                                    required:
                                    - queue-id
                                    type: object
                                  type: array
                              type: object
                          type: object
                        sflow:
                          description: InterfaceSflow struct
                          properties:
                            admin-state:
                              enum:
                              - disable
                              - enable
                              type: string
                          type: object
                        transceiver:
                          description: InterfaceTransceiver struct
                          properties:
                            ddm-events:
                              type: boolean
                            forward-error-correction:
                              default: disabled
                              enum:
                              - base-r
                              - disabled
                              - rs-108
                              - rs-528
                              - rs-544
                              type: string
                            tx-laser:
                              type: boolean
                          type: object
                        vlan-tagging:
                          type: boolean
                      required:
                      - name
                      type: object
                    type: array
                required:
                - interface
                type: object
              validationDetails:
                additionalProperties:
                  description: ValidationDetails provides the status of the configuration
                    applied on this particular device
                  properties:
                    leafRefParh:
                      description: LeafRefPath points to the leafref object value
                      type: string
                    leafRefValues:
                      description: LeafRefValues identifies the values assigned in
                        the leaf ref
                      items:
                        type: string
                      type: array
                    values:
                      description: Values identifies the object value that should
                        match the leafref value if empty it means the object does
                        not exist.
                      items:
                        type: string
                      type: array
                  type: object
                description: ConfigurationDependencyValidationDetails defines the
                  validation details of the resource object
                type: object
            type: object
        type: object
    served: true
    storage: true
    subresources:
      status: {}
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []

@lblackstone
Copy link
Member

Unless I'm mistaken, this is likely more complicated than "just" fixing the crd2pulumi tool (I tried).

Quite possibly. I haven't dug deep on the specifics here, but can think of a couple options that may be relevant.

  1. Here is an example where we replace a property name in the schema so that it doesn't contain invalid characters for C#. This will be language-specific, but may be a targeted way to fix in crd2pulumi without requiring external changes.
  2. The schema can be extended to support additional language-specific overrides. We use that today for import aliases and namespaces, but there's no reason we couldn't also support type name mapping. This would require crd2pulumi to specify the override in the schema (along these lines), and then would require changes in the pulumi/pulumi codegen to make use of these overrides. It's simple in concept, but would require a fair bit of work.

@danylo-omelchenko
Copy link

Hi, crd2pulumi is funtastic tool, but faced same issue with not supported generated names with dashes (-) in them. Is there any news on that topic?

@mattolenik mattolenik added the area/codegen Affects quality or correctness of generated code label Jun 2, 2022
@doy-materialize
Copy link

i'm also currently running into this problem, with python, rather than go - it's generating code like class CiliumEndpointStatusExternal-Identifiers(dict):. are there any updates here?

@mattolenik
Copy link
Contributor

Hi @doy-materialize, would it be possible to get the full CRD so that I can reproduce your problem?

@doy-materialize
Copy link

sure!

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  creationTimestamp: "2022-08-08T20:44:50Z"
  generation: 1
  labels:
    io.cilium.k8s.crd.schema.version: 1.24.3
  name: ciliumendpoints.cilium.io
  resourceVersion: "568"
  uid: 87f61f4e-c20b-4f50-a749-4a60e68662dd
spec:
  conversion:
    strategy: None
  group: cilium.io
  names:
    kind: CiliumEndpoint
    listKind: CiliumEndpointList
    plural: ciliumendpoints
    shortNames:
    - cep
    - ciliumep
    singular: ciliumendpoint
  scope: Namespaced
  versions:
  - additionalPrinterColumns:
    - description: Cilium endpoint id
      jsonPath: .status.id
      name: Endpoint ID
      type: integer
    - description: Cilium identity id
      jsonPath: .status.identity.id
      name: Identity ID
      type: integer
    - description: Ingress enforcement in the endpoint
      jsonPath: .status.policy.ingress.enforcing
      name: Ingress Enforcement
      type: boolean
    - description: Egress enforcement in the endpoint
      jsonPath: .status.policy.egress.enforcing
      name: Egress Enforcement
      type: boolean
    - description: Status of visibility policy in the endpoint
      jsonPath: .status.visibility-policy-status
      name: Visibility Policy
      type: string
    - description: Endpoint current state
      jsonPath: .status.state
      name: Endpoint State
      type: string
    - description: Endpoint IPv4 address
      jsonPath: .status.networking.addressing[0].ipv4
      name: IPv4
      type: string
    - description: Endpoint IPv6 address
      jsonPath: .status.networking.addressing[0].ipv6
      name: IPv6
      type: string
    name: v2
    schema:
      openAPIV3Schema:
        description: CiliumEndpoint is the status of a Cilium policy rule.
        properties:
          apiVersion:
            description: 'APIVersion defines the versioned schema of this representation
              of an object. Servers should convert recognized schemas to the latest
              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
            type: string
          kind:
            description: 'Kind is a string value representing the REST resource this
              object represents. Servers may infer this from the endpoint the client
              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
            type: string
          metadata:
            type: object
          status:
            description: EndpointStatus is the status of a Cilium endpoint.
            properties:
              controllers:
                description: Controllers is the list of failing controllers for this
                  endpoint.
                items:
                  description: ControllerStatus is the status of a failing controller.
                  properties:
                    configuration:
                      description: Configuration is the controller configuration
                      properties:
                        error-retry:
                          description: Retry on error
                          type: boolean
                        error-retry-base:
                          description: 'Base error retry back-off time Format: duration'
                          format: int64
                          type: integer
                        interval:
                          description: 'Regular synchronization interval Format: duration'
                          format: int64
                          type: integer
                      type: object
                    name:
                      description: Name is the name of the controller
                      type: string
                    status:
                      description: Status is the status of the controller
                      properties:
                        consecutive-failure-count:
                          format: int64
                          type: integer
                        failure-count:
                          format: int64
                          type: integer
                        last-failure-msg:
                          type: string
                        last-failure-timestamp:
                          type: string
                        last-success-timestamp:
                          type: string
                        success-count:
                          format: int64
                          type: integer
                      type: object
                    uuid:
                      description: UUID is the UUID of the controller
                      type: string
                  type: object
                type: array
              encryption:
                description: Encryption is the encryption configuration of the node
                properties:
                  key:
                    description: Key is the index to the key to use for encryption
                      or 0 if encryption is disabled.
                    type: integer
                type: object
              external-identifiers:
                description: ExternalIdentifiers is a set of identifiers to identify
                  the endpoint apart from the pod name. This includes container runtime
                  IDs.
                properties:
                  container-id:
                    description: ID assigned by container runtime
                    type: string
                  container-name:
                    description: Name assigned to container
                    type: string
                  docker-endpoint-id:
                    description: Docker endpoint ID
                    type: string
                  docker-network-id:
                    description: Docker network ID
                    type: string
                  k8s-namespace:
                    description: K8s namespace for this endpoint
                    type: string
                  k8s-pod-name:
                    description: K8s pod name for this endpoint
                    type: string
                  pod-name:
                    description: K8s pod for this endpoint(Deprecated, use K8sPodName
                      and K8sNamespace instead)
                    type: string
                type: object
              health:
                description: Health is the overall endpoint & subcomponent health.
                properties:
                  bpf:
                    description: bpf
                    type: string
                  connected:
                    description: Is this endpoint reachable
                    type: boolean
                  overallHealth:
                    description: overall health
                    type: string
                  policy:
                    description: policy
                    type: string
                type: object
              id:
                description: ID is the cilium-agent-local ID of the endpoint.
                format: int64
                type: integer
              identity:
                description: Identity is the security identity associated with the
                  endpoint
                properties:
                  id:
                    description: ID is the numeric identity of the endpoint
                    format: int64
                    type: integer
                  labels:
                    description: Labels is the list of labels associated with the
                      identity
                    items:
                      type: string
                    type: array
                type: object
              log:
                description: Log is the list of the last few warning and error log
                  entries
                items:
                  description: "EndpointStatusChange Indication of a change of status
                    \n swagger:model EndpointStatusChange"
                  properties:
                    code:
                      description: 'Code indicate type of status change Enum: [ok
                        failed]'
                      type: string
                    message:
                      description: Status message
                      type: string
                    state:
                      description: state
                      type: string
                    timestamp:
                      description: Timestamp when status change occurred
                      type: string
                  type: object
                type: array
              named-ports:
                description: "NamedPorts List of named Layer 4 port and protocol pairs
                  which will be used in Network Policy specs. \n swagger:model NamedPorts"
                items:
                  description: "Port Layer 4 port / protocol pair \n swagger:model
                    Port"
                  properties:
                    name:
                      description: Optional layer 4 port name
                      type: string
                    port:
                      description: Layer 4 port number
                      type: integer
                    protocol:
                      description: 'Layer 4 protocol Enum: [TCP UDP ICMP ICMPV6 ANY]'
                      type: string
                  type: object
                type: array
              networking:
                description: Networking is the networking properties of the endpoint.
                properties:
                  addressing:
                    description: IP4/6 addresses assigned to this Endpoint
                    items:
                      description: AddressPair is is a par of IPv4 and/or IPv6 address.
                      properties:
                        ipv4:
                          type: string
                        ipv6:
                          type: string
                      type: object
                    type: array
                  node:
                    description: NodeIP is the IP of the node the endpoint is running
                      on. The IP must be reachable between nodes.
                    type: string
                required:
                - addressing
                type: object
              policy:
                description: EndpointPolicy represents the endpoint's policy by listing
                  all allowed ingress and egress identities in combination with L4
                  port and protocol.
                properties:
                  egress:
                    description: EndpointPolicyDirection is the list of allowed identities
                      per direction.
                    properties:
                      adding:
                        description: Deprecated
                        items:
                          description: IdentityTuple specifies a peer by identity,
                            destination port and protocol.
                          properties:
                            dest-port:
                              type: integer
                            identity:
                              format: int64
                              type: integer
                            identity-labels:
                              additionalProperties:
                                type: string
                              type: object
                            protocol:
                              type: integer
                          type: object
                        type: array
                      allowed:
                        description: AllowedIdentityList is a list of IdentityTuples
                          that species peers that are allowed.
                        items:
                          description: IdentityTuple specifies a peer by identity,
                            destination port and protocol.
                          properties:
                            dest-port:
                              type: integer
                            identity:
                              format: int64
                              type: integer
                            identity-labels:
                              additionalProperties:
                                type: string
                              type: object
                            protocol:
                              type: integer
                          type: object
                        type: array
                      denied:
                        description: DenyIdentityList is a list of IdentityTuples
                          that species peers that are denied.
                        items:
                          description: IdentityTuple specifies a peer by identity,
                            destination port and protocol.
                          properties:
                            dest-port:
                              type: integer
                            identity:
                              format: int64
                              type: integer
                            identity-labels:
                              additionalProperties:
                                type: string
                              type: object
                            protocol:
                              type: integer
                          type: object
                        type: array
                      enforcing:
                        type: boolean
                      removing:
                        description: Deprecated
                        items:
                          description: IdentityTuple specifies a peer by identity,
                            destination port and protocol.
                          properties:
                            dest-port:
                              type: integer
                            identity:
                              format: int64
                              type: integer
                            identity-labels:
                              additionalProperties:
                                type: string
                              type: object
                            protocol:
                              type: integer
                          type: object
                        type: array
                    required:
                    - enforcing
                    type: object
                  ingress:
                    description: EndpointPolicyDirection is the list of allowed identities
                      per direction.
                    properties:
                      adding:
                        description: Deprecated
                        items:
                          description: IdentityTuple specifies a peer by identity,
                            destination port and protocol.
                          properties:
                            dest-port:
                              type: integer
                            identity:
                              format: int64
                              type: integer
                            identity-labels:
                              additionalProperties:
                                type: string
                              type: object
                            protocol:
                              type: integer
                          type: object
                        type: array
                      allowed:
                        description: AllowedIdentityList is a list of IdentityTuples
                          that species peers that are allowed.
                        items:
                          description: IdentityTuple specifies a peer by identity,
                            destination port and protocol.
                          properties:
                            dest-port:
                              type: integer
                            identity:
                              format: int64
                              type: integer
                            identity-labels:
                              additionalProperties:
                                type: string
                              type: object
                            protocol:
                              type: integer
                          type: object
                        type: array
                      denied:
                        description: DenyIdentityList is a list of IdentityTuples
                          that species peers that are denied.
                        items:
                          description: IdentityTuple specifies a peer by identity,
                            destination port and protocol.
                          properties:
                            dest-port:
                              type: integer
                            identity:
                              format: int64
                              type: integer
                            identity-labels:
                              additionalProperties:
                                type: string
                              type: object
                            protocol:
                              type: integer
                          type: object
                        type: array
                      enforcing:
                        type: boolean
                      removing:
                        description: Deprecated
                        items:
                          description: IdentityTuple specifies a peer by identity,
                            destination port and protocol.
                          properties:
                            dest-port:
                              type: integer
                            identity:
                              format: int64
                              type: integer
                            identity-labels:
                              additionalProperties:
                                type: string
                              type: object
                            protocol:
                              type: integer
                          type: object
                        type: array
                    required:
                    - enforcing
                    type: object
                type: object
              state:
                description: State is the state of the endpoint.
                enum:
                - creating
                - waiting-for-identity
                - not-ready
                - waiting-to-regenerate
                - regenerating
                - restoring
                - ready
                - disconnecting
                - disconnected
                - invalid
                type: string
              visibility-policy-status:
                type: string
            type: object
        required:
        - metadata
        type: object
    served: true
    storage: true
    subresources: {}
status:
  acceptedNames:
    kind: CiliumEndpoint
    listKind: CiliumEndpointList
    plural: ciliumendpoints
    shortNames:
    - cep
    - ciliumep
    singular: ciliumendpoint
  conditions:
  - lastTransitionTime: "2022-08-08T20:44:50Z"
    message: no conflicts found
    reason: NoConflicts
    status: "True"
    type: NamesAccepted
  - lastTransitionTime: "2022-08-08T20:44:50Z"
    message: the initial names have been accepted
    reason: InitialNamesAccepted
    status: "True"
    type: Established
  storedVersions:
  - v2

@stack72 stack72 added this to the 0.77 milestone Aug 19, 2022
@mattolenik mattolenik modified the milestones: 0.77, 0.78 Sep 6, 2022
mattolenik pushed a commit that referenced this issue Sep 20, 2022
The fix for the root cause is in pulumi/pulumi. Updating this repo will
pull in that fix. Added regression tests and verified that they fail
against master.
@lukehoban lukehoban modified the milestones: 0.78, 0.79 Oct 6, 2022
bors bot added a commit to pulumi/pulumi that referenced this issue Oct 11, 2022
10738: Go and Python codegen support symbols with hyphens in their names r=iwahbe a=mattolenik

# Description

This change enables the fix for pulumi/crd2pulumi#43, wherein hyphens `-` were mishandled and led to generation of invalid Go and Python. For example, code such as `func My-Thing() {}` or `class My-Thing`.

This change improves the existing casing functions and extracts them to a new subpackage, `cgstrings`.

Resolves pulumi/crd2pulumi#43. When this change is merged `go get -u` will fix the crd2pulumi issue.

## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my feature works


Co-authored-by: Matthew Olenik <molenik@pulumi.com>
Co-authored-by: Ian Wahbe <ian@wahbe.com>
bors bot added a commit to pulumi/pulumi that referenced this issue Oct 17, 2022
11049: Go & Python handle hypen in sdk gen (cgstrings take 2) r=iwahbe a=iwahbe

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

Fixes pulumi/crd2pulumi#43

## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [X] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [X] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Ian Wahbe <ian@wahbe.com>
bors bot added a commit to pulumi/pulumi that referenced this issue Oct 17, 2022
11049: Go & Python handle hyphen in sdk gen (cgstrings take 2) r=iwahbe a=iwahbe

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

Fixes pulumi/crd2pulumi#43

## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [X] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [X] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Ian Wahbe <ian@wahbe.com>
bors bot added a commit to pulumi/pulumi that referenced this issue Oct 17, 2022
11049: Go & Python handle hyphen in sdk gen (cgstrings take 2) r=iwahbe a=iwahbe

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

Fixes pulumi/crd2pulumi#43

## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [X] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [X] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


11055: feat(programtest): Enable ProgramTest for Azure-Native split modules r=AaronFriel a=AaronFriel

Unblocks Azure Native usage of ProgramTests in Go, resolves #11050's example by permitting usage such as:

```go
func genReplace(t *testing.T, subPkg string) string {
	moduleDir, err := filepath.Abs(fmt.Sprintf("../pulumi-azure-native-sdk/%s", subPkg))
	require.NoError(t, err)
	return fmt.Sprintf("github.com/pulumi/pulumi-azure-native-sdk/%s=%s", subPkg, moduleDir)
}


func getGoBaseOptions(t *testing.T) integration.ProgramTestOptions {
	base := getBaseOptions(t)
	baseGo := base.With(integration.ProgramTestOptions{
		Dependencies: map[string]string{
			genReplace("aad"),
		},
	})

	return baseGo
}
```

Resolves #11050



Co-authored-by: Ian Wahbe <ian@wahbe.com>
Co-authored-by: Aaron Friel <mayreply@aaronfriel.com>
@lblackstone lblackstone self-assigned this Oct 17, 2022
lblackstone pushed a commit that referenced this issue Oct 18, 2022
The fix for the root cause is in pulumi/pulumi. Updating this repo will
pull in that fix. Added regression tests and verified that they fail
against master.
@lkt82
Copy link

lkt82 commented Jan 28, 2024

Hi

I am getting this error for C# so the issue does not seam to be fixed.

@pierskarsenbarg
Copy link
Member

Re-opening because this is not fixed.

Reproduction:

  1. Create new folder and change to it: mkdir crdtest && cd cdrtest
  2. Run crd2pulumi pointing at this gist (yaml taken from this comment): crd2pulumi --dotnetPath dotnet --dotnet https://gist.githubusercontent.com/pierskarsenbarg/707236cbfd649025494aa010d95ffb93/raw/f5adab7fccd3007ecf21d675033a3c257c42b4fb/crd.yaml
  3. Change to dotnet folder: cd dotnet
  4. Fix Pulumi version in csproj (see this issue: Pulumi version in generated csproj file incorrect #130 )
  5. Build dotnet project: dotnet build to see error.

@lukehoban lukehoban added needs-triage Needs attention from the triage team and removed resolution/fixed This issue was fixed labels Mar 26, 2024
@pierskarsenbarg
Copy link
Member

From a very basic overview (running the crd2pulumi with different flags) it seems that only .NET and NodeJS are affected here. Python and Go seem fine

@rquitales
Copy link
Contributor

rquitales commented Apr 1, 2024

It looks like we'd need the fixes in upstream pu/pu as well for .Net, nodejs, an Java. Note, Java codegen was only recently enabled in this project, but a release has not been cut yet that includes this feature. A local build of crd2pulumi also encountered issues with Java generation.

Looking at the history for fixing this, pulumi/pulumi#11049 was merged that closed this issue. However, this upstream pu/pu fix only targeted Python and Go. We'd need something similar for the other languages.

@blampe
Copy link
Contributor

blampe commented Apr 8, 2024

It looks like we'd need the fixes in upstream pu/pu as well for .Net, nodejs, an Java. Note, Java codegen was only recently enabled in this project, but a release has not been cut yet that includes this feature. A local build of crd2pulumi also encountered issues with Java generation.

Looking at the history for fixing this, pulumi/pulumi#11049 was merged that closed this issue. However, this upstream pu/pu fix only targeted Python and Go. We'd need something similar for the other languages.

Created an issue on pu/pu here pulumi/pulumi#15874

I tried to get a minimal repro with just a failing schema, and that did reproduce the issue for me in ts/cs but I didn't see an issue with Java. It's unclear if this is due to my repro not capturing something specific to crd2pulumi; maybe the issue was fixed in a newer pulumi-java release than we're using; or something else.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/codegen Affects quality or correctness of generated code kind/bug Some behavior is incorrect or out of spec
Projects
None yet