Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MediaLive waitUntilInputDeleted does not work. #5125

Open
cobar79 opened this issue Apr 19, 2024 · 6 comments
Open

MediaLive waitUntilInputDeleted does not work. #5125

cobar79 opened this issue Apr 19, 2024 · 6 comments
Assignees
Labels
bug This issue is a bug. p2 This is a standard priority issue response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days.

Comments

@cobar79
Copy link

cobar79 commented Apr 19, 2024

Describe the bug

When using MediaLive waitUntilInputDeleted when the input is in a "Detached" state, the input is not deleted and the request times out.

When directly deleting the mediaLiveClient.deleteInput(DeleteInputRequest.builder().inputId(inputId).build()); the input is deleted.

However, this makes the deletion of the Input Security Group fail because it is not Idle.
I did search for MediaLiveWaiter waitUntilInputSecurityGroupIdle but it does not seem to exist.

Expected Behavior

The MediaLive input should be immediately deleted since it is in a Detached state and not time out. This would insure that the Security Group is Idle and able to be deleted.

Current Behavior

software.amazon.awssdk.core.exception.SdkClientException: The waiter has exceeded the max retry attempts: 21

Subsequently, you have to enter the console and delete the detached Input, wait a few seconds and delete the Security group once it is Idle.

Reproduction Steps

See

	public DescribeInputResponse deleteInput(String inputId) {
		//Deleting it without wait works, but leaves security group since it is not idle and there isn't
		//a waiter for deleting security group until idle.
//		mediaLiveClient.deleteInput(DeleteInputRequest.builder().inputId(inputId).build());

		DescribeInputResponse describeInputResponse = waitTillInputDetached(inputId);
		log.info("{}:: input state {}", describeInputResponse.id(), describeInputResponse.state());
		DescribeInputResponse deleteResponse;
		try (MediaLiveWaiter mlWaiter = MediaLiveWaiter.builder()
			.client(mediaLiveClient)
			.overrideConfiguration(override -> override
				.waitTimeout(Duration.ofSeconds(120))
				.maxAttempts(21)
				.build())
			.build()
		) {
			ResponseOrException<DescribeInputResponse> responseOrException = mlWaiter
				.waitUntilInputDeleted(builder -> builder.inputId(inputId))
				.matched();
			deleteResponse = responseOrException.response()
				.orElseThrow(() -> new VideoServiceRuntimeException(inputId + ":: not deleted", ErrorReason.SERVICE_EXCEPTION));
			log.info("{}:: input deleted", inputId);
			return deleteResponse;
		}
	}

Run MediaLiveServiceTests whenCreateChannel_givenChannelRequest_thenCreateChannel

Failure occurs because it is unclear where to set the Archive S3 URL in the SDK.
Archive group destination A
Enter a destination for archiving your first group.
URL s3://<bucket>/<folder>/<file-prefix>

		S3Uri s3Uri = s3Service.createArchiveFolder(liveName);
		//Add liveName to simulate s3://bucket/folder/liveName_NameModifier.segment#.ts)
		String destinationUri = s3Uri.uri().toString() + '/' + liveName;
		String[] finalDestinationUri = {destinationUri};

		return OutputGroup.builder()
			.name("archive-s3")
			.outputGroupSettings(outputGroupSettings -> outputGroupSettings
				.archiveGroupSettings(archiveGroupSettings -> archiveGroupSettings
					.rolloverInterval(mediaLiveConfig.getRolloverSeconds())
					.destination(destination -> destination.destinationRefId(finalDestinationUri[0]))
					.build())
				.build())
			.outputs(archiveOutput)
			.build();

Object references undefined destination "s3://bucket/mli-test-01-test-01/mli-test-01-test-01" (Service: MediaLive, Status Code: 422, Request ID: scoobyDubbyDoo)

Video Service App GitHub

Possible Solution

Fix waitUntilInputDeleted to delete the Detached Input

ResponseOrException<DescribeInputResponse> responseOrException = mlWaiter
				.waitUntilInputDeleted(builder -> builder.inputId(inputId))
				.matched();

Add waitUntilInputSecurityGroupIdle functionallity

Additional Information/Context

No response

AWS Java SDK version used

2.25.14

JDK version used

Corretto-17.0.7.7.1 (build 17.0.7+7-LTS)

Operating System and version

Windows 10

@cobar79 cobar79 added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Apr 19, 2024
@debora-ito
Copy link
Member

@cobar79 to clarify, your ask is for the waiter waitUntilInputDeleted to call DeleteInput under the hood? If so, that's not what waiters are designed for. Waiters will poll the resource to check if it reached a desired state but won't effectively initiate the state change itself. So you need to call DeleteInput and use the waiter afterwards.

Let me know if I'm missing something.

@debora-ito debora-ito added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days. and removed needs-triage This issue or PR still needs to be triaged. labels Apr 22, 2024
@debora-ito debora-ito self-assigned this Apr 22, 2024
@debora-ito debora-ito added the p2 This is a standard priority issue label Apr 23, 2024
@cobar79
Copy link
Author

cobar79 commented Apr 23, 2024

@debora-ito
To me it appears waitUntilInputDelete is broken in that it times out even though the input is detached.
I say this because I can delete without waiting. However, that just causes the deletion of the InputSecurityGroup to fail since the InputSecurityGroup is not "Idle". Since there is no waiting for the InputSecurityGroup to become "Idle" in the SDK, it has to rely on the "Input.waitUntilInputDelete" to be detached and deleted and the hopefully the security group to be Idle. I say hopefully, because I can delete the input in the Console and bring up the security group and it still be enabled for a second or two before becoming Idle and I can delete it.

@github-actions github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days. label Apr 23, 2024
@debora-ito
Copy link
Member

Ok, so I think what's missing here is a waiter for Input Security Group. 'waitUntilInputDelete' seems to be working as expected, its success condition is DELETED, not DETACHED, so the waiter will keep calling DescribeInput while the status is different than DELETED, and time out in this case.

According to the possible values of InputState, after calling DeleteInput it can move from DETACHED to DELETING (even very briefly) then to DELETED. That's why this waiter exists, the input can be on DELETING state for a while.

As you noticed, there's no waiters for Input Security Groups, so as a workaround you can poll it manually - here's a very simplistic example:

    int attemptNumber = 0;

    //manually polling InputSecurityGroup
    while (attemptNumber < 20) {
        DescribeInputSecurityGroupResponse describeResponse =
                mediaLiveClient.describeInputSecurityGroup(s-> s.inputSecurityGroupId(INPUT_SECGROUP_ID));
        if (describeResponse.state().equals(InputSecurityGroupState.IDLE)) {
            break;
        }
        Thread.sleep(500);
        attemptNumber++;
    }
    
    mediaLiveClient.deleteInputSecurityGroup(d->d.inputSecurityGroupId(INPUT_SECGROUP_ID));

In the meantime, I'll reach out to the MediaLive team and ask them to add waiters for Input Security Groups to the model.

Let us know if this helps.

@cobar79
Copy link
Author

cobar79 commented Apr 26, 2024

Agree that there should be a waiter on Input Security Group. I still believe the Input Waiter is broken since I always get
software.amazon.awssdk.core.exception.SdkClientException: The waiter has exceeded the max retry attempts: 21
Even after overriding the defaults

			.overrideConfiguration(override -> override
				.waitTimeout(Duration.ofSeconds(120))
				.maxAttempts(21)
				.build())

I didn't attempt to override the backoff strategy.

Will try the workaround but Thread.sleep tends to be an issue and ignored which is how I found the built in wait which seems to handle any interrupt issues.

@cobar79
Copy link
Author

cobar79 commented May 9, 2024

@debora-ito

I implemented the manual wait on the input security group. However, it is won't be reached or work since the input delete fails maxAttempts. This should not be failing since the input is already Detached? It takes very little time to delete it in the console.

In fact I bumped the waitTimeout to 3 minutes, jumped in the console and manually deleted the input and the waiter finished and deleted the input security group via the SDK.

software.amazon.awssdk.core.exception.SdkClientException: The waiter has exceeded the max retry attempts: 21

	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111)
	at software.amazon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:43)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutorHelper.nextDelayOrUnretryableException(WaiterExecutorHelper.java:77)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:88)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.maybeRetry(WaiterExecutor.java:100)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.evaluate(WaiterExecutor.java:75)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.doExecute(WaiterExecutor.java:60)
	at software.amazon.awssdk.core.internal.waiters.WaiterExecutor.execute(WaiterExecutor.java:48)
	at software.amazon.awssdk.core.internal.waiters.DefaultWaiter.run(DefaultWaiter.java:49)
	at software.amazon.awssdk.services.medialive.waiters.DefaultMediaLiveWaiter.waitUntilInputDeleted(DefaultMediaLiveWaiter.java:186)
	at software.amazon.awssdk.services.medialive.waiters.MediaLiveWaiter.waitUntilInputDeleted(MediaLiveWaiter.java:399)
	at com.bar.osi.video.service.impl.MediaLiveServiceImpl.waitTillInputDeleted(MediaLiveServiceImpl.java:243)
	at com.bar.osi.video.service.impl.MediaLiveServiceImpl.deleteInput(MediaLiveServiceImpl.java:223)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1092)
	at com.bar.osi.video.service.impl.MediaLiveServiceImpl.deleteInputSecurityGroup(MediaLiveServiceImpl.java:280)
	at com.bar.osi.video.service.impl.MediaLiveServiceImpl.cleanUp(MediaLiveServiceImpl.java:316)
	at com.bar.osi.video.service.impl.MediaLiveServiceImpl.createChannel(MediaLiveServiceImpl.java:215)
	at com.bar.osi.video.service.impl.MediaLiveServiceTests.whenCreateChannel_givenChannelRequest_thenCreateChannel(MediaLiveServiceTests.java:80)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

@debora-ito
Copy link
Member

In fact I bumped the waitTimeout to 3 minutes, jumped in the console and manually deleted the input and the waiter finished and deleted the input security group via the SDK.

To confirm I understand the steps you are taking:

(1) Input is in DETACHED state
(2) You call mediaLiveClient.deleteInput() using the SDK
(3) You call mlWaiter.waitUntilInputDeleted(), but it fails with maxAttempts reached

?

@debora-ito debora-ito added the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days. label May 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. p2 This is a standard priority issue response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days.
Projects
None yet
Development

No branches or pull requests

2 participants