AWS XRay SDK无法读取Docker容器中的环境变量

2019年7月10日 8点热度 0条评论

AWS XRay是一种跟踪服务,使您可以跟踪分布式系统中的请求,甚至可以分析您的服务。无需过多研究XRay的工作原理,它基本上可以监视您的服务,并通过UDP将有关每个请求的数据发送到该服务的守护程序,该守护程序收集此数据并将其发送到AWS。

当在本地或EC2中运行时,此守护程序位于运行该服务的计算机本地,并且在端口2000上可用。这是守护程序主机位置的默认配置。

在Kubernetes中运行时,您需要设置一个守护程序以在每个节点上运行。按照documentation for setting up XRay with Kubernetes,可以通过为所需主机设置环境变量AWS_XRAY_DAEMON_ADDRESS来覆盖默认值,或者可以设置JVM系统变量com.amazonaws.xray.emitters.daemonAddressSDK documentation中也有对此的引用。

由于我的用例以及我们如何在组织中共享配置,我想利用设置环境变量的方法。

根据文档,我们通过头盔图表将其设置为部署:

env:
  - name: AWS_XRAY_DAEMON_ADDRESS
    value: aws-xray-daemon.default

通过执行运行服务的Pod,并运行
printenv,我们可以看到此值在部署时已成功设置。

问题:

当XRay尝试剖析并将数据发送到守护程序时,会抛出
SdkClientException

com.amazonaws.SdkClientException: Unable to execute HTTP request: Connect to 127.0.0.1:2000 [/127.0.0.1] failed: Connection refused (Connection refused)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1201) ~[aws-java-sdk-core-1.11.739.jar!/:na]
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1147) ~[aws-java-sdk-core-1.11.739.jar!/:na]
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:796) ~[aws-java-sdk-core-1.11.739.jar!/:na]
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:764) ~[aws-java-sdk-core-1.11.739.jar!/:na]
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:738) ~[aws-java-sdk-core-1.11.739.jar!/:na]
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:698) ~[aws-java-sdk-core-1.11.739.jar!/:na]
        at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:680) ~[aws-java-sdk-core-1.11.739.jar!/:na]
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:544) ~[aws-java-sdk-core-1.11.739.jar!/:na]
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:524) ~[aws-java-sdk-core-1.11.739.jar!/:na]
        at com.amazonaws.services.xray.AWSXRayClient.doInvoke(AWSXRayClient.java:1607) ~[aws-java-sdk-xray-1.11.739.jar!/:na]
        at com.amazonaws.services.xray.AWSXRayClient.invoke(AWSXRayClient.java:1574) ~[aws-java-sdk-xray-1.11.739.jar!/:na]
        at com.amazonaws.services.xray.AWSXRayClient.invoke(AWSXRayClient.java:1563) ~[aws-java-sdk-xray-1.11.739.jar!/:na]
        at com.amazonaws.services.xray.AWSXRayClient.executeGetSamplingRules(AWSXRayClient.java:800) ~[aws-java-sdk-xray-1.11.739.jar!/:na]
        at com.amazonaws.services.xray.AWSXRayClient.getSamplingRules(AWSXRayClient.java:771) ~[aws-java-sdk-xray-1.11.739.jar!/:na]
        at com.amazonaws.xray.strategy.sampling.pollers.RulePoller.pollRule(RulePoller.java:65) ~[aws-xray-recorder-sdk-core-2.4.0.jar!/:na]
        at com.amazonaws.xray.strategy.sampling.pollers.RulePoller.lambda$start$0(RulePoller.java:46) ~[aws-xray-recorder-sdk-core-2.4.0.jar!/:na]
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:na]
        at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) ~[na:na]
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) ~[na:na]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:na]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:na]
        at java.base/java.lang.Thread.run(Unknown Source) ~[na:na]
        ...

这意味着AWS开发工具包不会按照文档建议使用此环境变量,而仅使用默认值
127.0.0.1:2000

然后,我深入研究了SDK代码,以了解如何检索此变量,并发现运行它的代码使用
System.getenv("AWS_XRAY_DAEMON_ADDRESS"),如下所示:

    /**
     * Environment variable key used to override the address to which UDP packets will be emitted. Valid values are of the form `ip_address:port`. Takes precedence over any system property,
     * constructor value, or setter value used.
     */
    public static final String DAEMON_ADDRESS_ENVIRONMENT_VARIABLE_KEY = "AWS_XRAY_DAEMON_ADDRESS";

    /**
     * System property key used to override the address to which UDP packets will be emitted. Valid values are of the form `ip_address:port`. Takes precedence over any constructor or setter value
     * used.
     */
    public static final String DAEMON_ADDRESS_SYSTEM_PROPERTY_KEY = "com.amazonaws.xray.emitters.daemonAddress";

    public DaemonConfiguration() {
        String environmentAddress = System.getenv(DAEMON_ADDRESS_ENVIRONMENT_VARIABLE_KEY);
        String systemAddress = System.getProperty(DAEMON_ADDRESS_SYSTEM_PROPERTY_KEY);

        if (setUDPAndTCPAddress(environmentAddress)) {
            logger.info(String.format("Environment variable %s is set. Emitting to daemon on address %s.", DAEMON_ADDRESS_ENVIRONMENT_VARIABLE_KEY, getUDPAddress()));
        } else if (setUDPAndTCPAddress(systemAddress)) {
            logger.info(String.format("System property %s is set. Emitting to daemon on address %s.", DAEMON_ADDRESS_SYSTEM_PROPERTY_KEY, getUDPAddress()));
        }
    }

所以我想,也许我没有正确设置环境变量?因此,我在启动服务时添加了获取环境变量的日志,发现JVM确实可以找到该值:

码:

System.out.println("System.getenv(\"AWS_XRAY_DAEMON_ADDRESS\")" + " = " + System.getenv("AWS_XRAY_DAEMON_ADDRESS")) 

输出:

System.getenv("AWS_XRAY_DAEMON_ADDRESS") = aws-xray-daemon.default

据我所知,此代码与AWS开发工具包应运行的代码完全匹配,但似乎从未执行过,而且如果执行,则其结果与我用我的测试结果不同日志。

在本地运行,我无法复制此问题,因为它会从本地环境变量中获取我提供的主机。我还确认了使用断点在本地运行时可以到达上面粘贴的AWS开发工具包代码。

有任何想法吗?

摇篮片段:

ext {
    ...
    springCloudVersion = "Greenwich.RELEASE"
    awsCoreVersion = '1.11.739'
    awsXrayVersion = '2.4.0' 
    ...
}

dependencyManagement {
    imports {
        mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}"
        mavenBom "com.amazonaws:aws-java-sdk-bom:${awsCoreVersion}"
        mavenBom "com.amazonaws:aws-xray-recorder-sdk-bom:${awsXrayVersion}"
    }
}

dependencies {
    ...

    implementation "com.amazonaws:aws-java-sdk-core"
    implementation "com.amazonaws:aws-xray-recorder-sdk-core" 
    implementation "com.amazonaws:aws-xray-recorder-sdk-aws-sdk" 
    implementation "com.amazonaws:aws-xray-recorder-sdk-spring" 
    implementation "com.amazonaws:aws-xray-recorder-sdk-apache-http" 
    implementation "com.amazonaws:aws-xray-recorder-sdk-sql-postgres" 

    implementation 'org.springframework.boot:spring-boot-starter-web'
    implementation 'org.springframework.boot:spring-boot-starter'
    implementation 'org.springframework.boot:spring-boot-starter-data-jpa'
    implementation 'org.springframework.boot:spring-boot-starter-security'

    ...

}

其他资讯:

在Spring Boot v2.2.1中运行

OpenJDK v11.0.4

摇篮v6.0.1

其他尝试:

-我尝试通过
Dockerfile设置环境变量。结果相同。

解决方案如下:

原来,我链接的blog post并不是一篇不错的博客文章。在示例中,他们未指定主机的端口:

env:
- name: AWS_XRAY_DAEMON_ADDRESS 
  value: xray-service.default

更改环境变量以包括端口可解决此问题:

env:
- name: AWS_XRAY_DAEMON_ADDRESS 
  value: xray-service.default:2000