Skip to content

Commit 36431cd

Browse files
ciroqueciroque
authored
Iss 16 oss friendly readme (#43)
* Merging deocumentation changes * CHECKPOINT - More tweaking * CHECKPOINT - More tweaking * CHECKPOINT - More tweaking * Updates README * Updates the original README * Updates links in the READM * Add diagram to main README * header alignment
1 parent ec6ac4f commit 36431cd

File tree

4 files changed

+523
-66
lines changed

4 files changed

+523
-66
lines changed

‎README.md

+114-66
Original file line numberDiff line numberDiff line change
@@ -1,112 +1,160 @@
1-
# nginx-k8s-loadbalancer
1+
<div style="margin-bottom: 5em;">
2+
<span>
3+
<img style="float: left;" src="nkl-logo.svg" width="124" />
4+
<h2 style="padding: 1.5em">nginx-k8s-loadbalancer</h2>
5+
</span>
6+
</div>
27

3-
<br/>
48

5-
# Welcome to the Nginx Kubernetes Load Balancer Solution!
9+
The NGINX K8s Loadbalancer, or _NKL_, is a Kubernetes controller that provides TCP load balancing external to a Kubernetes cluster running on-premise.
610

7-
<br/>
11+
## Requirements
812

9-
![Nginx K8s LB](docs/media/nkl-logo.png) | ![Nginx K8s LB](docs/media/nginx-2020.png)
10-
--- | ---
13+
[//]: # (### Who needs NKL?)
1114

12-
<br/>
15+
[//]: # ()
16+
[//]: # (- [ ] If you find yourself living in a world where Kubernetes is running on-premise instead of a cloud provider, you might need NKL.)
1317

14-
This repo contains source code and documents for a new `Kubernetes Controller from Nginx`, that provides TCP and HTTP load balancing external to a Kubernetes Cluster running On Premises.
18+
[//]: # (- [ ] If you want exceptional, best-in-class load-balancing for your Kubernetes clusters by using NGINX Plus, you might need NKL.)
1519

16-
<br/>
20+
[//]: # (- [ ] If you want the ability to manage your load-balancing configuration with the same tools you use to manage your Kubernetes cluster, you might need NKL.)
1721

18-
>>**This is a replacement for a Cloud Providers `Service Type Loadbalancer`, that is not available for On Premises Kubernetes Clusters.**
22+
### What you will need
1923

20-
<br/>
21-
<br/>
24+
- [ ] A Kubernetes cluster running on-premise.
25+
- [ ] One or more NGINX Plus hosts running outside your Kubernetes cluster (NGINX Plus hosts must have the ability to route traffic to the cluster).
2226

27+
There is a more detailed [Installation Guide](docs/InstallationGuide.md) available in the `docs/` directory.
2328

24-
# Overview
29+
### Why NKL?
2530

26-
- `NKL - Nginx Kubernetes Loadbalancer` is a new K8s Controller from Nginx, that monitors specified K8s Services, and then sends API calls to an external Nginx Plus server to manage Nginx Upstream servers dynamically.
27-
- This will `synchronize` the K8s Service Endpoint list, with the Nginx LB Server's upstream list.
28-
- The primary use case and Solution provided is for tracking the K8s` NodePort` IP:Port definitions for the Nginx Ingress Controller's `nginx-ingress Service`.
29-
- NKL is a native Kubernetes Controller, running, configured and managed with standard K8s commands.
30-
- NKL paired with the Nginx Plus Server located external to the K8s cluster, this new controller LB function will provide a `TCP Load Balancer Service` for On Premises K8s clusters, which do not have access to a Cloud providers "Service Type LoadBalancer".
31-
- NKL paired with the Nginx Plus Server located external to the Cluster, using Nginx's advanced HTTP features, provide an `HTTP Load Balancer Service` for Enterprise traffic management solutions, such as:
32-
- MultiCluster Active/Active Load Balancing
33-
- Horizontal Cluster Scaling
34-
- HTTP Split Clients - for A/B, Blue/Green, and Canary test and production traffic steering. Allows Cluster operations/maintainence like upgrades, patching, expansion and troubleshooting with no downtime or reloads
35-
- Advanced TLS Processing - MutualTLS, OCSP, FIPS, dynamic cert loading
36-
- Advanced Security features - Oauth, JWT, App Protect WAF Firewall, Rate and Bandwidth limits
37-
- Nginx Java Script (NJS) for custom solutions
38-
- Nginx Zone Sync of KeyVal data
31+
NKL provides a simple, easy-to-manage way to automate load balancing for your Kubernetes applications by leveraging NGINX Plus hosts running outside your cluster.
3932

40-
<br/>
33+
NKL installs easily, has a small footprint, and is easy to configure and manage.
4134

42-
## NKL Controller Software Design Overview - How it works
35+
NKL does not require learning a custom object model, you only have to understand NGINX configuration to get the most out of this solution.
36+
There is thorough documentation available with the specifics in the `docs/` directory.
4337

44-
[NKL Controller DESIGN and Architecture](DESIGN.md)
38+
### What does NKL do?
4539

46-
<br/>
40+
tl;dr:
4741

48-
## Reference Diagram for NKL TCP Load Balancer Service
42+
_**NKL is a Kubernetes controller that monitors Services and Nodes in your cluster, and then sends API calls to an external NGINX Plus server to manage NGINX Plus Upstream servers automatically.**_
4943

50-
<br/>
44+
That's all well and good, but what does it mean? Kubernetes clusters require some tooling to handling routing traffic from the outside world (e.g.: the Internet, corporate network, etc.) to the cluster.
45+
This is typically done with a load balancer. The load balancer is responsible for routing traffic to the appropriate worker node which then forwards the traffic to the appropriate Service / Pod.
5146

52-
![NKL Stream Diagram](docs/media/nkl-stream-diagram.png)
47+
If you are using a hosted web solution -- Digital Ocean, AWS, Azure, etc. -- you can use the cloud provider's load balancer service. Those services will create a load balancer for you.
48+
You can use the cloud provider's API to manage the load balancer, or you can use the cloud provider's web console.
5349

54-
<br/>
50+
If you are running Kubernetes on-premise and will need to manage your own load balancer, NKL can help.
5551

56-
## Sample Screenshots of Solution at Runtime
52+
NKL itself does not perform load balancing. Rather, NKL allows you to manage Service resources within your cluster to update your load balancers, with tooling you are most likely already using.
5753

58-
<br/>
54+
<img src="docs/media/nkl-blog-diagram-v1.png" width="768" />
5955

60-
![NGINX LB ConfigMap](docs/media/nkl-configmap.png)
61-
### ConfigMap with 2 Nginx LB Servers defined for HA
56+
## Getting Started
6257

63-
<br/>
58+
There are few bits of administrivia to get out of the way before you can start leveraging NKL for your load balancing needs.
6459

65-
![NGINX LB Create Nodeport](docs/media/nkl-stream-create-nodeport.png)
66-
### Nginx LB Server Dashboard, NodePort, and NKL Controller Logging
60+
As noted above, NKL is intended for when you have one or more Kubernetes clusters running on-premise. In addition to this,
61+
you need to have at least one NGINX Plus host running outside your cluster (Please refer to the [Roadmap](#Roadmap) for information about other load balancer servers).
6762

68-
### Legend:
69-
- Red - kubectl nodeport commands
70-
- Blue - nodeport and upstreams for http traffic
71-
- Indigo - nodeport and upstreams for https traffic
72-
- Green - NKL log for api calls to LB Server #1
73-
- Orange - Nginx LB Server upstream dashboard details
74-
- Kubernetes Worker Nodes are 10.1.1.8 and 10.1.1.10
63+
### Deployment
7564

76-
<br/>
65+
#### RBAC
7766

78-
The `Installation Guide` for TCP Loadbalancer Solution is located in the docs/tcp folder:
67+
As with everything Kubernetes, NKL requires RBAC permissions to function properly. The necessary resources are defined in the various YAML files in `deployement/rbac/`.
7968

80-
[TCP Installation Guide](docs/tcp/tcp-installation-guide.md)
69+
For convenience, two scripts are included, `apply.sh`, and `unapply.sh`. These scripts will apply or remove the RBAC resources, respectively.
8170

82-
<br/>
71+
The permissions required by NKL are modest. NKL requires the ability to read Resources via shared informers; the resources are Services, Nodes, and ConfigMaps.
72+
The Services and ConfigMap are restricted to a specific namespace (default: "nkl"). The Nodes resource is cluster-wide.
8373

84-
The `Installation Guide` for HTTP Loadbalancer Solution is located in the docs/http folder:
74+
#### Configuration
8575

86-
[HTTP Installation Guide](docs/http/http-installation-guide.md)
76+
NKL is configured via a ConfigMap, the default settings are found in `deployment/configmap.yaml`. Presently there is a single configuration value exposed in the ConfigMap, `nginx-hosts`.
77+
This contains a comma-separated list of NGINX Plus hosts that NKL will maintain.
8778

88-
<br/>
79+
You will need to update this ConfigMap to reflect the NGINX Plus hosts you wish to manage.
8980

90-
## Requirements
81+
If you were to deploy the ConfigMap and start NKL without updating the `nginx-hosts` value, don't fear; the ConfigMap resource is monitored for changes and NKL will update the NGINX Plus hosts accordingly when the resource is changed, no restart required.
82+
83+
There is an extensive [Installation Guide](docs/InstallationGuide.md) available in the `docs/` directory.
84+
Please refer to that for detailed instructions on how to deploy NKL and run a demo application.
85+
86+
#### Versioning
87+
88+
Versioning is a work in progress. The CI/CD pipeline is being developed and will be used to build and publish NKL images to the Container Registry.
89+
Once in place, semantic versioning will be used for published images.
90+
91+
#### Deployment Steps
92+
93+
To get NKL up and running in ten steps or fewer, follow these instructions (NOTE, all the aforementioned prerequisites must be met for this to work).
94+
There is a much more detailed [Installation Guide](docs/InstallationGuide.md) available in the `docs/` directory.
95+
96+
1. Clone this repo (optional, you can simply copy the `deployments/` directory)
97+
98+
```git clone git@github.com:nginxinc/nginx-k8s-loadbalancer.git```
99+
100+
2. Apply the RBAC resources
101+
102+
```./deployments/rbac/apply.sh```
103+
104+
3. Apply the Namespace
105+
106+
```kubectl apply -f deployments/namespace.yaml```
91107

92-
Please see the /docs folder and Installation Guides for detailed documentation.
108+
4. Update / Apply the ConfigMap (For best results update the `nginx-hosts` values first)
93109

94-
<br/>
110+
```kubectl apply -f deployments/configmap.yaml```
95111

96-
## Development
112+
5. Apply the Deployment
97113

98-
Read the [`CONTRIBUTING.md`](https://github.com/nginxinc/nginx-k8s-loadbalancer/blob/main/CONTRIBUTING.md) file.
114+
```kubectl apply -f deployments/deployment.yaml```
99115

100-
<br/>
116+
6. Check the logs
101117

102-
## Authors
103-
- Chris Akker - Solutions Architect - Community and Alliances @ F5, Inc.
104-
- Steve Wagner - Solutions Architect - Community and Alliances @ F5, Inc.
118+
```kubectl -n nkl get pods | grep nkl-deployment | cut -f1 -d" " | xargs kubectl logs -n nkl --follow $1```
105119

106-
<br/>
120+
At this point NKL should be up and running. Now would be a great time to go over to the [Installation Guide](docs/InstallationGuide.md)
121+
and follow the instructions to deploy a demo application.
122+
123+
### Monitoring
124+
125+
Presently NKL includes a fair amount of logging. This is intended to be used for debugging purposes.
126+
There are plans to add more robust monitoring and alerting in the future.
127+
128+
As a rule, we support the use of [OpenTelemetry](https://opentelemetry.io/) for observability, and we will be adding support in the near future.
129+
130+
## Contributing
131+
132+
Presently we are not accepting pull requests. However, we welcome your feedback and suggestions.
133+
Please open an issue to let us know what you think!
134+
135+
One way to contribute is to help us test NKL. We are looking for people to test NKL in a variety of environments.
136+
137+
If you are curious about the implementation, you should certainly browse the code, but first you might wish to refer to the [design document](docs/DESIGN.md).
138+
Some of the design decisions are explained there.
139+
140+
## Roadmap
141+
142+
While NKL was initially written specifically for NGINX Plus, we recognize there are other load-balancers that can be supported.
143+
144+
To this end, NKL has been architected to be extensible to support other "Border Servers".
145+
Border Servers are the term NKL uses to describe load-balancers, reverse proxies, etc. that run outside the cluster and handle
146+
routing outside traffic to your cluster.
147+
148+
While we have identified a few potential targets, we are open to suggestions. Please open an issue to share your thoughts on potential implementations.
149+
150+
We look forward to building a community around NKL and value all feedback and suggestions. Varying perspectives and embracing
151+
diverse ideas will be key to NKL becoming a solution that is useful to the community. We will consider it a success
152+
when we are able to accept pull requests from the community.
107153

108154
## License
109155

110156
[Apache License, Version 2.0](https://github.com/nginxinc/nginx-k8s-loadbalancer/blob/main/LICENSE)
111157

112-
&copy; [F5 Networks, Inc.](https://www.f5.com/) 2023
158+
&copy; [F5, Inc.](https://www.f5.com/) 2023
159+
160+
(but don't let that scare you, we're really nice people...)

‎DESIGN.md renamed to ‎docs/DESIGN.md

File renamed without changes.

‎docs/README.md

+112
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
# nginx-k8s-loadbalancer
2+
3+
<br/>
4+
5+
# Welcome to the Nginx Kubernetes Load Balancer Solution!
6+
7+
<br/>
8+
9+
![Nginx K8s LB](media/nkl-logo.png) | ![Nginx K8s LB](media/nginx-2020.png)
10+
--- | ---
11+
12+
<br/>
13+
14+
This repo contains source code and documents for a new `Kubernetes Controller from Nginx`, that provides TCP and HTTP load balancing external to a Kubernetes Cluster running On Premises.
15+
16+
<br/>
17+
18+
>>**This is a replacement for a Cloud Providers `Service Type Loadbalancer`, that is not available for On Premises Kubernetes Clusters.**
19+
20+
<br/>
21+
<br/>
22+
23+
24+
# Overview
25+
26+
- `NKL - Nginx Kubernetes Loadbalancer` is a new K8s Controller from Nginx, that monitors specified K8s Services, and then sends API calls to an external Nginx Plus server to manage Nginx Upstream servers dynamically.
27+
- This will `synchronize` the K8s Service Endpoint list, with the Nginx LB Server's upstream list.
28+
- The primary use case and Solution provided is for tracking the K8s` NodePort` IP:Port definitions for the Nginx Ingress Controller's `nginx-ingress Service`.
29+
- NKL is a native Kubernetes Controller, running, configured and managed with standard K8s commands.
30+
- NKL paired with the Nginx Plus Server located external to the K8s cluster, this new controller LB function will provide a `TCP Load Balancer Service` for On Premises K8s clusters, which do not have access to a Cloud providers "Service Type LoadBalancer".
31+
- NKL paired with the Nginx Plus Server located external to the Cluster, using Nginx's advanced HTTP features, provide an `HTTP Load Balancer Service` for Enterprise traffic management solutions, such as:
32+
- MultiCluster Active/Active Load Balancing
33+
- Horizontal Cluster Scaling
34+
- HTTP Split Clients - for A/B, Blue/Green, and Canary test and production traffic steering. Allows Cluster operations/maintainence like upgrades, patching, expansion and troubleshooting with no downtime or reloads
35+
- Advanced TLS Processing - MutualTLS, OCSP, FIPS, dynamic cert loading
36+
- Advanced Security features - Oauth, JWT, App Protect WAF Firewall, Rate and Bandwidth limits
37+
- Nginx Java Script (NJS) for custom solutions
38+
- Nginx Zone Sync of KeyVal data
39+
40+
<br/>
41+
42+
## NKL Controller Software Design Overview - How it works
43+
44+
[NKL Controller DESIGN and Architecture](DESIGN.md)
45+
46+
<br/>
47+
48+
## Reference Diagram for NKL TCP Load Balancer Service
49+
50+
<br/>
51+
52+
![NKL Stream Diagram](media/nkl-blog-diagram-v1.png)
53+
54+
<br/>
55+
56+
## Sample Screenshots of Solution at Runtime
57+
58+
<br/>
59+
60+
![NGINX LB ConfigMap](media/nkl-configmap.png)
61+
### ConfigMap with 2 Nginx LB Servers defined for HA
62+
63+
<br/>
64+
65+
![NGINX LB Create Nodeport](media/nkl-stream-create-nodeport.png)
66+
### Nginx LB Server Dashboard, NodePort, and NKL Controller Logging
67+
68+
### Legend:
69+
- Red - kubectl nodeport commands
70+
- Blue - nodeport and upstreams for http traffic
71+
- Indigo - nodeport and upstreams for https traffic
72+
- Green - NKL log for api calls to LB Server #1
73+
- Orange - Nginx LB Server upstream dashboard details
74+
- Kubernetes Worker Nodes are 10.1.1.8 and 10.1.1.10
75+
76+
<br/>
77+
78+
The `Installation Guide` for TCP Loadbalancer Solution is located in the tcp folder:
79+
80+
[TCP Installation Guide](tcp/tcp-installation-guide.md)
81+
82+
<br/>
83+
84+
The `Installation Guide` for HTTP Loadbalancer Solution is located in the http folder:
85+
86+
[HTTP Installation Guide](http/http-installation-guide.md)
87+
88+
<br/>
89+
90+
## Requirements
91+
92+
Please see the /docs folder and Installation Guides for detailed documentation.
93+
94+
<br/>
95+
96+
## Development
97+
98+
Read the [`CONTRIBUTING.md`](https://github.com/nginxinc/nginx-k8s-loadbalancer/blob/main/CONTRIBUTING.md) file.
99+
100+
<br/>
101+
102+
## Authors
103+
- Chris Akker - Solutions Architect - Community and Alliances @ F5, Inc.
104+
- Steve Wagner - Solutions Architect - Community and Alliances @ F5, Inc.
105+
106+
<br/>
107+
108+
## License
109+
110+
[Apache License, Version 2.0](https://github.com/nginxinc/nginx-k8s-loadbalancer/blob/main/LICENSE)
111+
112+
&copy; [F5 Networks, Inc.](https://www.f5.com/) 2023

0 commit comments

Comments
 (0)