Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restore support of single-port Che mode (on docker infra) #8471

Merged
merged 20 commits into from
Jan 31, 2018
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion dockerfiles/cli/images.template
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
IMAGE_INIT=${BUILD_ORGANIZATION}/${BUILD_PREFIX}-init:${BUILD_TAG}
IMAGE_CHE=${BUILD_ORGANIZATION}/${BUILD_PREFIX}-server:${BUILD_TAG}
IMAGE_COMPOSE=docker/compose:1.10.1
IMAGE_TRAEFIK=traefik:v1.3.0-rc3
IMAGE_TRAEFIK=traefik:v1.5.0
IMAGE_POSTGRES=centos/postgresql-96-centos7
IMAGE_KEYCLOACK=jboss/keycloak-openshift:3.3.0.CR2-3
2 changes: 1 addition & 1 deletion dockerfiles/cli/version/latest/images
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
IMAGE_INIT=eclipse/che-init:latest
IMAGE_CHE=eclipse/che-server:latest
IMAGE_COMPOSE=docker/compose:1.10.1
IMAGE_TRAEFIK=traefik:v1.3.0-rc3
IMAGE_TRAEFIK=traefik:v1.5.0
IMAGE_POSTGRES=centos/postgresql-96-centos7:9.6
IMAGE_KEYCLOACK=jboss/keycloak-openshift:3.3.0.CR2-3
2 changes: 1 addition & 1 deletion dockerfiles/cli/version/nightly/images
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
IMAGE_INIT=eclipse/che-init:nightly
IMAGE_CHE=eclipse/che-server:nightly
IMAGE_COMPOSE=docker/compose:1.10.1
IMAGE_TRAEFIK=traefik:v1.3.0-rc3
IMAGE_TRAEFIK=traefik:v1.5.0
IMAGE_POSTGRES=centos/postgresql-96-centos7:9.6
IMAGE_KEYCLOACK=jboss/keycloak-openshift:3.3.0.CR2-3
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import com.google.inject.AbstractModule;
import com.google.inject.assistedinject.FactoryModuleBuilder;
import com.google.inject.multibindings.Multibinder;
import org.eclipse.che.api.workspace.server.URLRewriter;
import org.eclipse.che.api.workspace.server.spi.RuntimeInfrastructure;
import org.eclipse.che.api.workspace.server.spi.provision.env.CheApiEnvVarProvider;
import org.eclipse.che.infrastructure.docker.client.DockerRegistryDynamicAuthResolver;
Expand All @@ -32,6 +33,8 @@
import org.eclipse.che.workspace.infrastructure.docker.provisioner.proxy.ProxySettingsProvisioner;
import org.eclipse.che.workspace.infrastructure.docker.provisioner.securityopt.SecurityOptProvisioner;
import org.eclipse.che.workspace.infrastructure.docker.provisioner.volume.ExtraVolumesProvisioner;
import org.eclipse.che.workspace.infrastructure.docker.server.mapping.ExternalIpURLRewriter;
import org.eclipse.che.workspace.infrastructure.docker.server.mapping.SinglePortUrlRewriter;

/** @author Alexander Garagatyi */
public class DockerInfraModule extends AbstractModule {
Expand Down Expand Up @@ -67,5 +70,11 @@ protected void configure() {
bind(
org.eclipse.che.workspace.infrastructure.docker.monit.DockerAbandonedResourcesCleaner
.class);

if (Boolean.parseBoolean(System.getenv("CHE_SINGLE_PORT"))) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest not to use conditional binding. We are usually complaining such solutions. Instead of that you can use technique that was used in Che 5 https://github.com/eclipse/che/blob/5.22.x/dockerfiles/init/modules/che/templates/che.env.erb#L81

bind(URLRewriter.class).to(SinglePortUrlRewriter.class);
} else {
bind(URLRewriter.class).to(ExternalIpURLRewriter.class);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,6 @@
import org.eclipse.che.workspace.infrastructure.docker.monit.AbnormalMachineStopHandler;
import org.eclipse.che.workspace.infrastructure.docker.monit.DockerMachineStopDetector;
import org.eclipse.che.workspace.infrastructure.docker.network.NetworkLifecycle;
import org.eclipse.che.workspace.infrastructure.docker.server.mapping.ExternalIpURLRewriter;
import org.slf4j.Logger;

/**
Expand Down Expand Up @@ -86,7 +85,7 @@ public class DockerInternalRuntime extends InternalRuntime<DockerRuntimeContext>
public DockerInternalRuntime(
@Assisted DockerRuntimeContext context,
@Assisted List<Warning> warnings,
ExternalIpURLRewriter urlRewriter,
URLRewriter urlRewriter,
NetworkLifecycle networks,
DockerMachineStarter machineStarter,
EventService eventService,
Expand Down Expand Up @@ -119,7 +118,7 @@ public DockerInternalRuntime(
@Assisted DockerRuntimeContext context,
@Assisted List<ContainerListEntry> containers,
@Assisted List<Warning> warnings,
ExternalIpURLRewriter urlRewriter,
URLRewriter urlRewriter,
NetworkLifecycle networks,
DockerMachineStarter machineStarter,
EventService eventService,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,13 @@
import javax.inject.Named;
import org.eclipse.che.api.core.ValidationException;
import org.eclipse.che.api.core.model.workspace.runtime.RuntimeIdentity;
import org.eclipse.che.api.workspace.server.URLRewriter;
import org.eclipse.che.api.workspace.server.spi.InfrastructureException;
import org.eclipse.che.api.workspace.server.spi.InternalInfrastructureException;
import org.eclipse.che.api.workspace.server.spi.RuntimeContext;
import org.eclipse.che.infrastructure.docker.client.json.ContainerListEntry;
import org.eclipse.che.workspace.infrastructure.docker.container.DockerContainers;
import org.eclipse.che.workspace.infrastructure.docker.model.DockerEnvironment;
import org.eclipse.che.workspace.infrastructure.docker.server.mapping.ExternalIpURLRewriter;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

Expand All @@ -38,7 +38,7 @@ public class DockerRuntimeContext extends RuntimeContext<DockerEnvironment> {

private static final Logger LOG = LoggerFactory.getLogger(DockerRuntimeContext.class);

private final ExternalIpURLRewriter urlRewriter;
private final URLRewriter urlRewriter;
private final String websocketOutputEndpoint;
private final DockerRuntimeFactory runtimeFactory;
private final DockerContainers containers;
Expand All @@ -54,7 +54,7 @@ public DockerRuntimeContext(
DockerContainers containers,
DockerSharedPool sharedPool,
RuntimeConsistencyChecker consistencyChecker,
ExternalIpURLRewriter urlRewriter,
URLRewriter urlRewriter,
@Named("che.websocket.endpoint") String cheWebsocketEndpoint)
throws InfrastructureException, ValidationException {

Expand All @@ -70,7 +70,7 @@ public DockerRuntimeContext(
@Override
public URI getOutputChannel() throws InfrastructureException {
try {
return URI.create(urlRewriter.rewriteURL(getIdentity(), null, websocketOutputEndpoint));
return URI.create(urlRewriter.rewriteURL(getIdentity(), null, null, websocketOutputEndpoint));
} catch (IllegalArgumentException ex) {
throw new InternalInfrastructureException(
"Failed to get the output channel because: " + ex.getLocalizedMessage());
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
package org.eclipse.che.workspace.infrastructure.docker.local;

import javax.inject.Inject;
import javax.inject.Named;
import javax.inject.Singleton;
import org.eclipse.che.api.core.model.workspace.runtime.RuntimeIdentity;
import org.eclipse.che.api.workspace.server.spi.InfrastructureException;
Expand All @@ -23,6 +24,7 @@
import org.eclipse.che.workspace.infrastructure.docker.provisioner.ContainerSystemSettingsProvisionersApplier;
import org.eclipse.che.workspace.infrastructure.docker.provisioner.env.EnvVarsConverter;
import org.eclipse.che.workspace.infrastructure.docker.provisioner.labels.RuntimeLabelsProvisioner;
import org.eclipse.che.workspace.infrastructure.docker.provisioner.labels.SinglePortLabelsProvisioner;
import org.eclipse.che.workspace.infrastructure.docker.provisioner.memory.MemoryAttributeConverter;
import org.eclipse.che.workspace.infrastructure.docker.provisioner.server.ServersConverter;
import org.eclipse.che.workspace.infrastructure.docker.provisioner.volume.VolumesConverter;
Expand All @@ -36,36 +38,42 @@
@Singleton
public class LocalCheDockerEnvironmentProvisioner implements DockerEnvironmentProvisioner {

private final boolean isSinglePortEnabled;
private final ContainerSystemSettingsProvisionersApplier dockerSettingsProvisioners;
private final BindMountProjectsVolumeProvisioner hostMountingProjectsVolumeProvisioner;
private final LocalInstallersBinariesVolumeProvisioner installersBinariesVolumeProvisioner;
private final RuntimeLabelsProvisioner runtimeLabelsProvisioner;
private final DockerApiHostEnvVariableProvisioner dockerApiEnvProvisioner;
private final WsAgentServerConfigProvisioner wsAgentServerConfigProvisioner;
private final SinglePortLabelsProvisioner singlePortLabelsProvisioner;
private final ServersConverter serversConverter;
private final EnvVarsConverter envVarsConverter;
private final MemoryAttributeConverter memoryAttributeConverter;
private final VolumesConverter volumesConverter;

@Inject
public LocalCheDockerEnvironmentProvisioner(
@Named("che.single.port") boolean isSinglePortEnabled,
ContainerSystemSettingsProvisionersApplier dockerSettingsProvisioners,
BindMountProjectsVolumeProvisioner hostMountingProjectsVolumeProvisioner,
LocalInstallersBinariesVolumeProvisioner installersBinariesVolumeProvisioner,
RuntimeLabelsProvisioner runtimeLabelsProvisioner,
DockerApiHostEnvVariableProvisioner dockerApiEnvProvisioner,
WsAgentServerConfigProvisioner wsAgentServerConfigProvisioner,
SinglePortLabelsProvisioner singlePortLabelsProvisioner,
ServersConverter serversConverter,
EnvVarsConverter envVarsConverter,
MemoryAttributeConverter memoryAttributeConverter,
VolumesConverter volumesConverter) {

this.isSinglePortEnabled = isSinglePortEnabled;
this.dockerSettingsProvisioners = dockerSettingsProvisioners;
this.hostMountingProjectsVolumeProvisioner = hostMountingProjectsVolumeProvisioner;
this.installersBinariesVolumeProvisioner = installersBinariesVolumeProvisioner;
this.runtimeLabelsProvisioner = runtimeLabelsProvisioner;
this.dockerApiEnvProvisioner = dockerApiEnvProvisioner;
this.wsAgentServerConfigProvisioner = wsAgentServerConfigProvisioner;
this.singlePortLabelsProvisioner = singlePortLabelsProvisioner;
this.serversConverter = serversConverter;
this.envVarsConverter = envVarsConverter;
this.memoryAttributeConverter = memoryAttributeConverter;
Expand All @@ -90,5 +98,8 @@ public void provision(DockerEnvironment internalEnv, RuntimeIdentity identity)
wsAgentServerConfigProvisioner.provision(internalEnv, identity);
dockerSettingsProvisioners.provision(internalEnv, identity);
dockerApiEnvProvisioner.provision(internalEnv, identity);
if (isSinglePortEnabled) {
singlePortLabelsProvisioner.provision(internalEnv, identity);
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
/*
* Copyright (c) 2012-2018 Red Hat, Inc.
* All rights reserved. This program and the accompanying materials
* are made available under the terms of the Eclipse Public License v1.0
* which accompanies this distribution, and is available at
* http://www.eclipse.org/legal/epl-v10.html
*
* Contributors:
* Red Hat, Inc. - initial API and implementation
*/
package org.eclipse.che.workspace.infrastructure.docker.provisioner.labels;

import static java.lang.String.format;

import java.util.HashMap;
import java.util.Map;
import javax.inject.Inject;
import javax.inject.Named;
import org.eclipse.che.api.core.model.workspace.config.ServerConfig;
import org.eclipse.che.api.core.model.workspace.runtime.RuntimeIdentity;
import org.eclipse.che.api.workspace.server.spi.InfrastructureException;
import org.eclipse.che.api.workspace.server.spi.environment.InternalMachineConfig;
import org.eclipse.che.commons.annotation.Nullable;
import org.eclipse.che.workspace.infrastructure.docker.model.DockerContainerConfig;
import org.eclipse.che.workspace.infrastructure.docker.model.DockerEnvironment;
import org.eclipse.che.workspace.infrastructure.docker.provisioner.ConfigurationProvisioner;
import org.eclipse.che.workspace.infrastructure.docker.server.mapping.SinglePortHostnameBuilder;

/**
* Sets necessary container labels for the single-port proxy (Traefik).
*
* @author Max Shaposhnik (mshaposh@redhat.com)
*/
public class SinglePortLabelsProvisioner implements ConfigurationProvisioner {

private final SinglePortHostnameBuilder hostnameBuilder;

@Inject
public SinglePortLabelsProvisioner(
@Nullable @Named("che.docker.ip.external") String externalIpOfContainers,
@Named("che.docker.ip") String internalIpOfContainers) {
this.hostnameBuilder =
new SinglePortHostnameBuilder(externalIpOfContainers, internalIpOfContainers);
}

@Override
public void provision(DockerEnvironment internalEnv, RuntimeIdentity identity)
throws InfrastructureException {
for (Map.Entry<String, InternalMachineConfig> machineEntry :
internalEnv.getMachines().entrySet()) {
final String machineName = machineEntry.getKey();
Map<String, String> containerLabels = new HashMap<>();
for (Map.Entry<String, ServerConfig> serverEntry :

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need traefik rules for internal servers

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added check

machineEntry.getValue().getServers().entrySet()) {
final String serverName = serverEntry.getKey().replace('/', '-');

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that server name can contain other symbols that are not allowed in the DNS entry. Can you check whether such a use case breaks this code?

final String host =
"Host:" + hostnameBuilder.build(serverName, machineName, identity.getWorkspaceId());
final String serviceName = machineName + "-" + serverName;
final String port = serverEntry.getValue().getPort().split("/")[0];

containerLabels.put(format("traefik.%s.port", serviceName), port);
containerLabels.put(format("traefik.%s.frontend.entryPoints", serviceName), "http");

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since single port mode effectively supports only HTTP protocol it may be useful to specify this in che.env and docs

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added note

containerLabels.put(format("traefik.%s.frontend.rule", serviceName), host);
// Needed to activate per-service rules
containerLabels.put("traefik.frontend.rule", machineName);
}
DockerContainerConfig dockerConfig = internalEnv.getContainers().get(machineName);
dockerConfig.getLabels().putAll(containerLabels);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,11 @@ public ExternalIpURLRewriter(
}

@Override
public String rewriteURL(@Nullable RuntimeIdentity identity, @Nullable String name, String url)
public String rewriteURL(
@Nullable RuntimeIdentity identity,
@Nullable String machineName,
@Nullable String serverName,
String url)
throws InfrastructureException {

if (externalIpOfContainers != null) {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
/*
* Copyright (c) 2012-2018 Red Hat, Inc.
* All rights reserved. This program and the accompanying materials
* are made available under the terms of the Eclipse Public License v1.0
* which accompanies this distribution, and is available at
* http://www.eclipse.org/legal/epl-v10.html
*
* Contributors:
* Red Hat, Inc. - initial API and implementation
*/
package org.eclipse.che.workspace.infrastructure.docker.server.mapping;

import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.StringJoiner;

/**
* Produces host names in form:
* server-<serverName>.<machineName>.<workspaceId>.<external_or_internal_address>.nip.io
*
* @author Max Shaposhnik (mshaposh@redhat.com)
*/
public class SinglePortHostnameBuilder {

private final String externalAddress;
private final String internalAddress;

public SinglePortHostnameBuilder(String externalAddress, String internalAddress) {
this.externalAddress = externalAddress;
this.internalAddress = internalAddress;
}

/**
* Constructs hostname from given params.
*
* @param serverName optional server name
* @param machineName optional machine name
* @param workspaceID optional workspace ID
* @return composite hostname
*/
public String build(String serverName, String machineName, String workspaceID) {
StringJoiner joiner = new StringJoiner(".");
if (serverName != null) {
joiner.add("server-" + serverName.replace('/', '-'));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if multiple dashes happen, is it valid host that works in single port case?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you elaborate on why this prefix server- is needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not necessary. Just to be more clear where the link points to.

}
if (machineName != null) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you describe in javadocs the case when some component is null?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

joiner.add(machineName);
}
if (workspaceID != null) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When any of arguments is null this method produces a hostname that differs from what is declared in the javadocs to this class. Are you sure that null arguments should be allowed?

joiner.add(workspaceID);
}
joiner.add(
externalAddress != null
? getWildcardNipDomain(externalAddress)
: getWildcardNipDomain(internalAddress));
return joiner.toString();
}

/**
* Gets a Wildcard domain based on the ip using an external provider nip.io
*
* @return wildcard domain
*/
private String getWildcardNipDomain(String localAddress) {
return String.format("%s.%s", getExternalIp(localAddress), "nip.io");

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since nip.io is not always available it would be nice to allow to override this with another hostname, such as xip.io or user specific.

}

private String getExternalIp(String localAddress) {
try {
return InetAddress.getByName(localAddress).getHostAddress();
} catch (UnknownHostException e) {
throw new UnsupportedOperationException(
"Unable to find the IP for the address '" + localAddress + "'", e);
}
}
}
Loading