Tags
It’s quite easy to use dockerhub with kubernetes deployments. Unfortunately that’s not the case if you want to use Amazon ECR as your container repository.
Amazon ECR has a few quirks before you can use it in your kubernetes platform. Amazon ECR does not allow you to use the API keys directly to create images. You need to generate a token to be used with ECR & the token is valid only for 12 hours.
Before you begin, please add below dependencies to your maven project.
<dependency>
<groupId>com.github.docker-java</groupId>
<artifactId>docker-java</artifactId>
<version>3.0.14</version>
</dependency>
<dependency>
<groupId>io.kubernetes</groupId>
<artifactId>client-java</artifactId>
<version>4.0.0-beta1</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-ecr</artifactId>
<version>1.11.477</version>
</dependency>
There are 2 main parts in using Amazon ECR with Docker & Kubernetes –
I) Push Docker Image to Amazon ECR
- Create Amazon ECR Authorization Token
@Value("${aws.ecr.access.key}") private String accessKey; @Value("${aws.ecr.access.secret}") private String accessSecret; @Value("${aws.ecr.default.region}") private String region; @Override public AuthorizationData getECRAuthorizationData(String repositoryName) { //Create AWS Credentials using Access Key AWSCredentials awsCredentials = new BasicAWSCredentials(accessKey, accessSecret); //Create AWS ECR Client AmazonECR amazonECR = AmazonECRClientBuilder.standard() .withRegion(region) .withCredentials(new AWSStaticCredentialsProvider(awsCredentials)) .build(); Repository repository = null; //Describe Repos. Check if repo exists for given repository name try { DescribeRepositoriesRequest describeRepositoriesRequest = new DescribeRepositoriesRequest(); List listOfRepos = new ArrayList(); listOfRepos.add(repositoryName); describeRepositoriesRequest.setRepositoryNames(listOfRepos); DescribeRepositoriesResult describeRepositoriesResult = amazonECR.describeRepositories(describeRepositoriesRequest); List repositories = describeRepositoriesResult.getRepositories(); repository = repositories.get(0); }catch(Exception e){ System.out.println("Error fetching repo. Error is : " + e.getMessage()); System.out.println("Creating repo...."); //Create Repository if required CreateRepositoryRequest createRepositoryRequest = new CreateRepositoryRequest().withRepositoryName(repositoryName); CreateRepositoryResult createRepositoryResult = amazonECR.createRepository(createRepositoryRequest); System.out.println("Created new repository : " + createRepositoryResult.getRepository().getRegistryId()); repository = createRepositoryResult.getRepository(); } //Get Auth Token for Repository using it's registry Id GetAuthorizationTokenResult authorizationToken = amazonECR .getAuthorizationToken(new GetAuthorizationTokenRequest().withRegistryIds(repository.getRegistryId())); List authorizationData = authorizationToken.getAuthorizationData(); return authorizationData.get(0); }
- Use token in docker java client
String userPassword = StringUtils.newStringUtf8(Base64.decodeBase64(authData.getAuthorizationToken())); String user = userPassword.substring(0, userPassword.indexOf(":")); String password = userPassword.substring(userPassword.indexOf(":") + 1); System.out.println("ECR Endpoint : " + authData.getProxyEndpoint()); //Create Docker Config DockerClientConfig config = DefaultDockerClientConfig.createDefaultConfigBuilder() .withDockerHost(dockerUrl) .withDockerTlsVerify(false) .withRegistryUrl(authData.getProxyEndpoint()) .withRegistryUsername(user) .withRegistryPassword(password) .withRegistryEmail("padmarag.lokhande@golaunchpad.io") .build(); DockerClient docker = DockerClientBuilder.getInstance(config).build();
- Build Docker image
String imageId = docker.buildImageCmd() .withDockerfile(new File(params.getFilePath() + "\\Dockerfile")) .withPull(true) .withNoCache(true) .withTag("latest") .exec(new BuildImageResultCallback()) .awaitImageId();
- Tag Docker image
String tag = "latest"; String repository = authData.getProxyEndpoint().replaceFirst("https://", org.apache.commons.lang.StringUtils.EMPTY) + "/" + params.getApplicationName(); TagImageCmd tagImageCmd = docker.tagImageCmd(imageId, repository, tag); tagImageCmd.exec();
- Push Docker image to Amazon ECR
docker.pushImageCmd(repository) .withTag("latest") .exec(new PushImageResultCallback()) .awaitCompletion(600, TimeUnit.SECONDS);
II) Pull Docker Image from ECR into Kubernetes
- Create Secret in Kubernetes
First we need to create Kubernetes config using kubernetes api server url and token for admin account.
ApiClient client = Config.fromToken(master, token,false);
String userPassword = org.apache.commons.codec.binary.StringUtils.newStringUtf8(Base64.decodeBase64(ecrAuthorizationData.getAuthorizationToken()));
String user = userPassword.substring(0, userPassword.indexOf(":"));
String password = userPassword.substring(userPassword.indexOf(":") + 1);
This is one important difference between normal kubernetes secret and special docker ecr secret. Check the type “kubernetes.io/dockerconfigjson”
V1Secret newSecret = new V1SecretBuilder()
.withNewMetadata()
.withName(ECR_REGISTRY)
.withNamespace(params.getNamespace())
.endMetadata()
.withType("kubernetes.io/dockerconfigjson")
.build();
newSecret.setType("kubernetes.io/dockerconfigjson");
The content for kubernetes docker secret needs to be created in specifically formatted json using data as set below. This is then set as byte data in V1Secret.
HashMap secretData = new HashMap(1);
String dockerCfg = String.format("{\"auths\": {\"%s\": {\"username\": \"%s\",\t\"password\": \"%s\",\"email\": \"%s\",\t\"auth\": \"%s\"}}}",
ecrAuthorizationData.getProxyEndpoint(),
user,
password,
"padmarag.lokhande@golaunchpad.io",
ecrAuthorizationData.getAuthorizationToken());
Map data = new HashMap();
data.put(".dockerconfigjson",dockerCfg.getBytes());
newSecret.setData(data);
V1Secret namespacedSecret = api.createNamespacedSecret(params.getNamespace(), newSecret, true, params.getPretty(), params.getDryRun());