martes, 26 de abril de 2016

Spring cloud netflix + Kubernetes + Docker

 

Introduction


The goal of this blog is know how to combine several technologies to make a  application oriented micro serivces, Automating deployment, operations, and scaling of containerized applications.

The example is composed of three parts:

1. A simple application to manage orders and lines, using spring rest.

2. Spring cloud netflix for managing microservices. We will use the following components:

Netflix Eureka - Service Discovery Server Netflix Eureka allows microservices to register themselves at runtime.

Netflix Ribbon - Dynamic Routing and Load Balancer Netflix Ribbon can be used by service consumers to lookup services at runtime. Ribbon uses the information available in Eureka to locate appropriate service instances. If more than one instance is found, Ribbon will apply load balancing to spread the requests over the available instances. Ribbon does not run as a separate service but instead as an embedded component in each service consumer.

Netflix Hystrix - Circuit breaker Netflix Hystrix provides circuit breaker capabilities to a service consumer. If a service doesn’t respond (e.g. due to a timeout or a communication error), Hystrix can redirect the call to an internal fallback method in the service consumer. If a service repeatedly fails to respond, Hystrix will open the circuit and fast fail (i.e. call the internal fallback method without trying to call the service) on every subsequent call until the service is available again. To determine wether the service is available again Hystrix allow some requests to try out the service even if the circuit is open. Hystrix executes embedded within its service consumer.

Netflix Hystrix dashboard and Netflix Turbine - Monitor Dashboard Hystrix dashboard can be used to provide a graphical overview of circuit breakers and Turbine can, based on information in Eureka, provide the dashboard with information from all circuit breakers in a system landscape.

3. Kubernetes is an open-source system for automating deployment, operations, and scaling of containerized applications using cluster system. Kubernetes allow run the application anywhere, giving you the freedom to take advantage of on-premise, hybrid, or public cloud infrastructure, letting you effortlessly move workloads to where it matters to you.

Recommended reading



Installation


Prerequisites



Installation


We Install docker. To install docker we follow the instructions on this page: https://docs.docker.com/engine/installation/linux/ubuntulinux/

We install kubernetes. There are many way to install kubernetes. I'm going to do locally. To install locally we follow the following document: http://kubernetes.io/v1.0/docs/getting-started-guides/locally.html

Tips to install kubernetes:

  1. Git clone in https://github.com/kubernetes/kubernetes
  2. Install etcd into /user/local
  3. Set path to go and etcd into sudo
  4. Build kubtctl (client tool) using “sudo build/run.sh hack/build-cross.sh”
  5. Copy the files generated for the corresponding platform from "_output/ dockerized" to a working folder and set path to this folder 


Project


Kubernete is based in docker images, We need create the application service into docker images, so we'll use gradle-docker plugin to build images.

We'll have add gradle docker dependence in build.gradle root and the docker plugin


also is necesary include the docker section, setting image base.

 

In each project must indicate the group, application name and the entry point for application execution from docker

 

The project structure is represented by 6 projects:




DiscoveryService: Service Discovery Server Netflix Eureka allows microservices to register themselves at runtime as they appear in the system landscape.

Is configured on 8761 port.

Kubernetes has its own discovery mechanism, but we will use the netflix. This allows us to manage centrally. We will use only the features container management that provides kubernetes. Kubernetes manages the drop and recovery nodes automatically.

Look all configuration at https://github.com/davsuapas/KubernetesMicroservices/blob/master/KubernetesMicroservices/DiscoveryService/src/main/resources/application.yml


EdgeService:
Edge Server Zuul is (of course) our gatekeeper to the outside world, not allowing any unauthorized external requests pass through. Zulu also provides a well known entry point to the microservices in the system landscape. Using dynamically allocated ports is convenient to avoid port conflicts and to minimize administration but it makes it of course harder for any given service consumer. Zuul uses Ribbon to lookup available services and routes the external request to an appropriate service instance.


Is configured on 8765 port.

The Order Service is only visible from outside and OrderLine is only visible from inside, so we write the following lines:

 



EdgeService need to know the discovery service address (DiscoveryService). This is configured by defining defaultzone:


DISCOVERY_SERVICE_HOST and DISCOVERY_SERVICE_PORT are environment variables that provides kubernetes to know the addresses of the services installed on the container (PODS). These variables are created in the system when services are created by kubernetes.

Usually a service in kubernetes is used like
an abstraction which defines a logical set of PODS and a policy by which to access them - sometimes called a micro-service.
As an example, consider an image-processing backend which is running with 3 replicas. Those replicas are fungible - frontends do not care which backend they use. While the actual PODS that compose the backend set may change, the frontend clients should not need to be aware of that or keep track of the list of backends themselves. The Service abstraction enables this decoupling.

I'll use kubernete service to know the address of discovery netflix service.




MonitorService: Monitor Dashboard Hystrix dashboard can be used to provide a graphical overview of circuit breakers and Turbine can, based on information in Eureka, provide the dashboard with information from all circuit breakers in a system landscape.


Is configured on 7979 port.




TurbineService: is an application that aggregates all of the relevant /hystrix.stream endpoints into a combined /turbine.stream for use in the Hystrix Dashboard. Individual instances are located via Eureka

Turbine need RabbitMQ for managing metrics. The same way that discovery service, i need to create a kubernetes service to know the address of the RabbitMQ pod. 



OrderService: This will be the service business that provides me orders across rest. This service will connect orderline service to get order lines. In case there is a problem with the orderline service, we will have a fallback method to return a result. Usually is used a database memory (redis) to retrieve information from cache.

Is configured on 8080 port.


   
OrderLineService: This service serve order lines to order service. The service is only visible from order service. Visibility is set up in EdgeService.


Is configured on 8080 port.




 

Kubernete set up

 

 

We need to create so pods as application services available, and also the kubernetes services discussed above, one for discovery service and other for RabbitMQ.

Kubernetes settings is through yaml files.




In kubernetes there are different types of pods. We will use three types:
  1. Pod: For each application. There is only one instance for this service type.
  2. Service: To discover pods. This type of pod is used to communicate several pods. Using environment variables or DNS
  3. ReplicationController: A pods that allows you to specify the number of instances reaised.
I will explain one of each type and most important properties:

Discover.yaml

 
kind: Pod type.
labels: It is the name that kubernetes service will seek to locate a pod.
image: The docker image built with gradle plugin.
ports: Here we will set the port that we have configured in each service.
 

Discoverservice.yaml


selector: Pod label name which will route requests


Order.yaml


 

replicas: Instances number to raised


Spring cloud netflix + Kubernetes + Docker. Part II 

2 comentarios:

  1. Thank you David for your interesting article. I tried to follow your strategy but i am facing a strange problem.
    When i tried to run Eureka with 3 replicas (under a kubernetes cluster with 2 nodes and 1 master), each eureka pod cannot identify its replicas, so when a microservice instance goes down, only one pod get notified and the other still seeing this instance as running.how may i workaround this issue ?

    ResponderEliminar
    Respuestas
    1. I understand that you need is run eureka in peer awareness mode. You can see documentation in http://cloud.spring.io/spring-cloud-static/spring-cloud-netflix/1.2.3.RELEASE/. Look at "peer awareness" section. On the other hand, all pods in a cluster can see each other without NAT. The way to connect your microservice with eureka is use kubernete service, as i comment in this article.

      Eliminar