Allow NGF to run on Openshift

[bjee19]

Add SecurityContextConstraints for NGF, documentation on additional steps when installing NGF on an Openshift environment.

Problem: NGF fails to deploy on Openshift.

Solution: Add SecurityContextConstraints for NGF, giving NGF to correct permissions to deploy on Openshift.

Testing: Manual deployment of NGF with NGINX and NGINX+. Also set up the cafe example and ensured traffic flowed.

After manual deployment of NGF, here is the cafe example directing traffic correctly:

[cloud-user@ocp-provisioner cafe-example]$ curl --resolve cafe.example.com:8080:127.0.0.1 http://cafe.example.com:8080/coffee
Handling connection for 8080
Server address: 10.131.0.49:8080
Server name: coffee-7b9b4bbd99-dtdzv
Date: 08/May/2024:23:11:50 +0000
URI: /coffee
Request ID: e49fc248ba007d069782e7afe43954b9
[cloud-user@ocp-provisioner cafe-example]$ curl --resolve cafe.example.com:8080:127.0.0.1 http://cafe.example.com:8080/tea
Handling connection for 8080
Server address: 10.131.0.50:8080
Server name: tea-7f5799695f-2wcnx
Date: 08/May/2024:23:12:04 +0000
URI: /tea
Request ID: 5728cac75d9c670b2ed3125b39181771

After manual deployment of NGF with NGINX+, here is the cafe example directing traffic correctly:

NAME                                        READY   STATUS    RESTARTS      AGE
ngf-nginx-gateway-fabric-586cdc4477-779l2   2/2     Running   1 (31s ago)   49s
[cloud-user@ocp-provisioner ~]$ kubectl logs -n nginx-gateway ngf-nginx-gateway-fabric-586cdc4477-779l2 --container nginx
2024/05/14 18:16:06 [notice] 22#22: using the "epoll" event method
2024/05/14 18:16:06 [notice] 22#22: nginx/1.25.3 (nginx-plus-r31-p1)

...

[cloud-user@ocp-provisioner ~]$ curl --resolve cafe.example.com:8080:127.0.0.1 http://cafe.example.com:8080/coffee
Server address: 10.128.2.28:8080
Server name: coffee-7b9b4bbd99-ltsmp
Date: 14/May/2024:20:36:02 +0000
URI: /coffee
Request ID: 5952f1e1a07070db43d1f07300d85cc4
[cloud-user@ocp-provisioner ~]$ curl --resolve cafe.example.com:8080:127.0.0.1 http://cafe.example.com:8080/tea
Server address: 10.131.0.28:8080
Server name: tea-7f5799695f-75vwz
Date: 14/May/2024:20:36:19 +0000
URI: /tea
Request ID: d233f1856fcf96e5b49684232257a7ec

Closes #1674

Checklist

Before creating a PR, run through this checklist and mark each as complete.

• I have read the CONTRIBUTING doc
• I have added tests that prove my fix is effective or that my feature works
• I have checked that all unit tests pass after adding my changes
• I have updated necessary documentation
• I have rebased my branch onto main
• I will ensure my PR is targeting the main branch and pulling from my branch from my own fork

Release notes

If this PR introduces a change that affects users and needs to be mentioned in the release notes,
please add a brief note that summarizes the change.

Add SecurityContextConstraints so NGF can run on Openshift.

[bjee19]

Documentation on additional steps when installing NGF on an Openshift environment is still in progress, would like some feedback on the current SCC file and the references documentation.

[bjee19]

Decided to split out the openshift-permissions / NGF permissions into a separate issue found here #1985 , will be resolving/closing all reviews dealing with that document.