Skip to content

feat(QTDI-1291): support guess schema as service #1220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ozhelezniak-talend wants to merge 2 commits into
base: master
Choose a base branch
from
Open

feat(QTDI-1291): support guess schema as service #1220

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
27971e9
feat(QTDI-1291): support guess schema as service
ozhelezniak-talend Sep 22, 2025
b43f88f
Merge branch 'master' into ozhelezniak/QTDI-1291_discoverschema_as_se…
ozhelezniak-talend Aug 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 12 additions & 7 deletions ...ent-api/src/main/java/org/talend/sdk/component/api/exception/DiscoverSchemaException.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,11 @@
import javax.json.bind.annotation.JsonbCreator;
import javax.json.bind.annotation.JsonbPropertyOrder;

import lombok.Data;
import lombok.EqualsAndHashCode;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import lombok.ToString;

/**
* This class is dedicated to Studio's guess schema feature.
Expand All @@ -29,7 +32,10 @@
*
* See me TCOMP-2342 for more details.
*/
@Data
@Setter
@Getter
@ToString
@NoArgsConstructor
@EqualsAndHashCode(callSuper = true)
@JsonbPropertyOrder({ "localizedMessage", "message", "stackTrace", "suppressed", "possibleHandleErrorWith" })
public class DiscoverSchemaException extends RuntimeException {
Expand Down Expand Up @@ -58,7 +64,7 @@ public enum HandleErrorWith {
* This won't query for any user input.
* When specifying this option, developer should be sure that no side effect can be generated by connector.
*/
EXECUTE_LIFECYCLE;
EXECUTE_LIFECYCLE
}

private HandleErrorWith possibleHandleErrorWith = HandleErrorWith.EXCEPTION;
Expand All @@ -69,20 +75,19 @@ public DiscoverSchemaException(final ComponentException e) {

public DiscoverSchemaException(final ComponentException e, final HandleErrorWith handling) {
super(e.getOriginalMessage(), e.getCause());
setPossibleHandleErrorWith(handling);
this.possibleHandleErrorWith = handling;
}

public DiscoverSchemaException(final String message, final HandleErrorWith handling) {
super(message);
setPossibleHandleErrorWith(handling);
this.possibleHandleErrorWith = handling;
}

@JsonbCreator
public DiscoverSchemaException(final String message, final StackTraceElement[] stackTrace,
final HandleErrorWith handling) {
super(message);
setStackTrace(stackTrace);
setPossibleHandleErrorWith(handling);
this.possibleHandleErrorWith = handling;
}

}
6 changes: 6 additions & 0 deletions ...r-model/src/main/java/org/talend/sdk/component/server/front/model/error/ErrorPayload.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,5 +28,11 @@ public class ErrorPayload {

private ErrorDictionary code;

private String subCode;

Comment on lines +31 to +32

@undx undx Jun 24, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've a model change, we need to specify it in jira issue (pinging @acatoire) but, apparently, it doesn't affect TCOMP-api-test.

@acatoire acatoire Aug 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the API test does not fail, this mean that this part was not covered or the test was not strict enough, to be checked.

@ozhelezniak-talend ozhelezniak-talend Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @lxia-talend
Could you handle that part?

@lxia-talend lxia-talend Aug 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have @DiscoverSchema and @DiscoverSchemaExtended api test case.

@ozhelezniak-talend ozhelezniak-talend Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And they are okay and we can merge this PR? =)

@lxia-talend lxia-talend Aug 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think we can.

private String description;

public ErrorPayload(final ErrorDictionary code, final String description) {
this(code, null, description);
}
}
55 changes: 38 additions & 17 deletions ...ponent-server/src/main/java/org/talend/sdk/component/server/front/ActionResourceImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@
import javax.ws.rs.core.Response;

import org.talend.sdk.component.api.exception.ComponentException;
import org.talend.sdk.component.api.exception.DiscoverSchemaException;
import org.talend.sdk.component.api.exception.DiscoverSchemaException.HandleErrorWith;
import org.talend.sdk.component.runtime.manager.ComponentManager;
import org.talend.sdk.component.runtime.manager.ContainerComponentRegistry;
import org.talend.sdk.component.runtime.manager.ServiceMeta;
Expand Down Expand Up @@ -175,15 +177,15 @@ private CompletableFuture<Response> doExecuteLocalAction(final String family, fi
// check org.talend.sdk.component.server.service.ComponentManagerService.readCurrentLocale if you change it
}, Runnable::run).exceptionally(e -> {
final Throwable cause;
if (e.getCause() instanceof ExecutionException) {
cause = e.getCause().getCause();
if (e.getCause() instanceof final ExecutionException exece) {
cause = exece.getCause();
} else {
cause = e.getCause();
}
if (cause instanceof WebApplicationException wae) {
final Response response = wae.getResponse();
String message = "";
if (wae.getResponse().getEntity() instanceof ErrorPayload) {
if (response.getEntity() instanceof ErrorPayload) {
throw wae; // already logged and setup broken so just rethrow
} else {
Comment thread
ozhelezniak-talend marked this conversation as resolved.
try {
Expand All @@ -210,31 +212,50 @@ private CompletableFuture<Response> doExecuteLocalAction(final String family, fi

private Response onError(final Throwable re) {
log.warn(re.getMessage(), re);
if (re.getCause() instanceof WebApplicationException wae) {
return wae.getResponse();
if (re instanceof final WebApplicationException webException) {
return webException.getResponse();
} else if (re.getCause() instanceof final WebApplicationException webException) {
return webException.getResponse();
}

if (re instanceof ComponentException ce) {
final String description = "Action execution failed with: " + ofNullable(re.getMessage())
.orElseGet(() -> re instanceof NullPointerException
? "unexpected null"
: "no error message");
if (re instanceof final DiscoverSchemaException eSchema) {
// we send reason to recognize the error on client side
final String subCode = ofNullable(eSchema.getPossibleHandleErrorWith())
.orElse(HandleErrorWith.EXCEPTION)
.toString();
Comment thread
ozhelezniak-talend marked this conversation as resolved.
throw new WebApplicationException(Response
.status(ce.getErrorOrigin() == ComponentException.ErrorOrigin.USER ? 400
: ce.getErrorOrigin() == ComponentException.ErrorOrigin.BACKEND ? 456 : 520,
"Unexpected callback error")
.entity(new ErrorPayload(ErrorDictionary.ACTION_ERROR,
"Action execution failed with: " + ofNullable(re.getMessage())
.orElseGet(() -> re instanceof NullPointerException ? "unexpected null"
: "no error message")))
.status(400, subCode)
.entity(new ErrorPayload(ErrorDictionary.ACTION_ERROR, subCode, description))
.build());
Comment thread
ozhelezniak-talend marked this conversation as resolved.
Comment thread
ozhelezniak-talend marked this conversation as resolved.
Comment thread
ozhelezniak-talend marked this conversation as resolved.
Comment thread
ozhelezniak-talend marked this conversation as resolved.
} else if (re instanceof final ComponentException eComponent) {
throw new WebApplicationException(Response
.status(evaluateStatusCodeForException(eComponent), "Unexpected callback error")
.entity(new ErrorPayload(ErrorDictionary.ACTION_ERROR, description))
.build());
Comment thread
ozhelezniak-talend marked this conversation as resolved.
Comment thread
ozhelezniak-talend marked this conversation as resolved.
}

throw new WebApplicationException(Response
.status(520, "Unexpected callback error")
.entity(new ErrorPayload(ErrorDictionary.ACTION_ERROR,
"Action execution failed with: " + ofNullable(re.getMessage())
.orElseGet(() -> re instanceof NullPointerException ? "unexpected null"
: "no error message")))
.entity(new ErrorPayload(ErrorDictionary.ACTION_ERROR, description))
.build());
Comment thread
ozhelezniak-talend marked this conversation as resolved.
Comment thread
ozhelezniak-talend marked this conversation as resolved.
}

private static int evaluateStatusCodeForException(final ComponentException eComponent) {
if (null == eComponent.getErrorOrigin()) {
return 520;
}

return switch (eComponent.getErrorOrigin()) {
case USER -> 400;
case BACKEND -> 456;
default -> 520;
};
}

private Stream<ActionItem> findVirtualActions(final Predicate<String> typeMatcher,
final Predicate<String> componentMatcher, final Locale locale) {
return virtualActions
Expand Down
160 changes: 99 additions & 61 deletions ...onent-server/src/main/java/org/talend/sdk/component/server/service/PropertiesService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
import java.util.Collection;
import java.util.Comparator;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
Expand All @@ -35,6 +36,7 @@
import javax.inject.Inject;
import javax.json.bind.Jsonb;

import org.talend.sdk.component.api.record.Schema;
import org.talend.sdk.component.runtime.internationalization.ParameterBundle;
import org.talend.sdk.component.runtime.manager.ParameterMeta;
import org.talend.sdk.component.runtime.manager.reflect.parameterenricher.ValidationParameterEnricher;
Expand Down Expand Up @@ -69,54 +71,89 @@ public Stream<SimplePropertyDefinition> buildProperties(final List<ParameterMeta

private Stream<SimplePropertyDefinition> buildProperties(final List<ParameterMeta> meta, final ClassLoader loader,
final Locale locale, final DefaultValueInspector.Instance rootInstance, final ParameterMeta parent) {
return meta.stream().flatMap(p -> {
final String path = sanitizePropertyName(p.getPath());
final String name = sanitizePropertyName(p.getName());
final String type = p.getType().name();
final boolean isEnum = p.getType() == ParameterMeta.Type.ENUM;
PropertyValidation validation = propertyValidationService.map(p.getMetadata());
if (isEnum) {
if (validation == null) {
validation = new PropertyValidation();
}
validation.setEnumValues(p.getProposals());
}
final Map<String, String> sanitizedMetadata = ofNullable(p.getMetadata())
.map(m -> m
.entrySet()
.stream()
.filter(e -> !e.getKey().startsWith(ValidationParameterEnricher.META_PREFIX))
.collect(toLinkedMap(e -> e.getKey().replace("tcomp::", ""), Map.Entry::getValue)))
.orElse(null);
final Map<String, String> metadata;
if (parent != null) {
metadata = sanitizedMetadata;
} else {
metadata = ofNullable(sanitizedMetadata).orElseGet(HashMap::new);
metadata.put("definition::parameter::index", String.valueOf(meta.indexOf(p)));
}
final DefaultValueInspector.Instance instance = defaultValueInspector
.createDemoInstance(
ofNullable(rootInstance).map(DefaultValueInspector.Instance::getValue).orElse(null), p);
final ParameterBundle bundle = p.findBundle(loader, locale);
final ParameterBundle parentBundle = parent == null ? null : parent.findBundle(loader, locale);
return Stream
.concat(Stream
.of(new SimplePropertyDefinition(path, name,
bundle.displayName(parentBundle).orElse(p.getName()), type, toDefault(instance, p),
validation, rewriteMetadataForLocale(metadata, parentBundle, bundle),
bundle.placeholder(parentBundle).orElse(p.getName()),
!isEnum ? null
: p
.getProposals()
.stream()
.collect(toLinkedMap(identity(),
key -> bundle
.enumDisplayName(parentBundle, key)
.orElse(key))))),
buildProperties(p.getNestedParameters(), loader, locale, instance, p));
}).sorted(Comparator.comparing(SimplePropertyDefinition::getPath)); // important cause it is the way you want to
// see it
return meta.stream()
.flatMap(p -> buildProperty(p, meta, loader, locale, rootInstance, parent))
// important cause it is the way you want to see it
.sorted(Comparator.comparing(SimplePropertyDefinition::getPath));
}

private Stream<SimplePropertyDefinition> buildProperty(final ParameterMeta p,
final List<ParameterMeta> siblings,
final ClassLoader loader,
final Locale locale,
final DefaultValueInspector.Instance rootInstance,
final ParameterMeta parent) {
final String path = sanitizePropertyName(p.getPath());
final String name = sanitizePropertyName(p.getName());
final String type = p.getType().name();

final PropertyValidation validation = buildValidation(p);
final Map<String, String> metadata = buildMetadata(p, siblings, parent);

final DefaultValueInspector.Instance instance = defaultValueInspector
.createDemoInstance(ofNullable(rootInstance)
.map(DefaultValueInspector.Instance::getValue)
.orElse(null), p);
final ParameterBundle bundle = p.findBundle(loader, locale);
final ParameterBundle parentBundle = parent == null ? null : parent.findBundle(loader, locale);
final String displayName = bundle.displayName(parentBundle).orElse(p.getName());
final String placeholder = bundle.placeholder(parentBundle).orElse(p.getName());

final LinkedHashMap<String, String> enumValues = buildEnumDisplayNames(p, bundle, parentBundle);
final SimplePropertyDefinition def = new SimplePropertyDefinition(path, name, displayName, type,
toDefault(instance, p), validation, rewriteMetadataForLocale(metadata, parentBundle, bundle),
placeholder, enumValues);

return Stream.concat(
Stream.of(def),
buildProperties(p.getNestedParameters(), loader, locale, instance, p));
}

private PropertyValidation buildValidation(final ParameterMeta p) {
PropertyValidation validation = propertyValidationService.map(p.getMetadata());
if (p.getType() != ParameterMeta.Type.ENUM) {
return validation;
}
if (validation == null) {
validation = new PropertyValidation();
}
validation.setEnumValues(p.getProposals());
return validation;
}

private Map<String, String> buildMetadata(
final ParameterMeta p,
final List<ParameterMeta> siblings,
final ParameterMeta parent) {
final Map<String, String> sanitized = ofNullable(p.getMetadata())
.map(m -> m
.entrySet()
.stream()
.filter(e -> !e.getKey().startsWith(ValidationParameterEnricher.META_PREFIX))
.collect(toLinkedMap(e -> e.getKey().replace("tcomp::", ""), Map.Entry::getValue)))
.orElse(null);
if (parent != null) {
return sanitized;
}

final Map<String, String> metadata = ofNullable(sanitized).orElseGet(HashMap::new);
metadata.put("definition::parameter::index", String.valueOf(siblings.indexOf(p)));
// this one to mark the Schema parameter somehow to differentiate it from the branch name
if (p.getJavaType() instanceof Class<?> clazzType && Schema.class.isAssignableFrom(clazzType)) {
metadata.put("definition::parameter::schema", "");
}
return metadata;
Comment thread
ozhelezniak-talend marked this conversation as resolved.
}

private LinkedHashMap<String, String> buildEnumDisplayNames(final ParameterMeta p, final ParameterBundle bundle,
final ParameterBundle parentBundle) {
if (p.getType() != ParameterMeta.Type.ENUM) {
return null;
}

return p.getProposals()
.stream()
.collect(toLinkedMap(identity(), key -> bundle.enumDisplayName(parentBundle, key).orElse(key)));
}

private Map<String, String> rewriteMetadataForLocale(final Map<String, String> metadata,
Expand Down Expand Up @@ -151,20 +188,21 @@ private Map<String, String> rewriteLayoutMetadata(final Map<String, String> meta
return metadata;
}
final Predicate<Map.Entry<String, ?>> shouldBeRewritten = k -> keysToRewrite.contains(k.getKey());
return Stream
.concat(metadata.entrySet().stream().filter(shouldBeRewritten.negate()),
metadata
.entrySet()
.stream()
.filter(shouldBeRewritten)
.map(it -> new AbstractMap.SimpleEntry<>(bundle
.gridLayoutName(parentBundle,
it
.getKey()
.substring("ui::gridlayout::".length(),
it.getKey().length() - "::value".length()))
return Stream.concat(
metadata.entrySet()
.stream()
.filter(shouldBeRewritten.negate()),
metadata.entrySet()
.stream()
.filter(shouldBeRewritten)
.map(it -> new AbstractMap.SimpleEntry<>(
bundle.gridLayoutName(parentBundle,
it.getKey()
.substring("ui::gridlayout::".length(),
it.getKey().length() - "::value".length()))
.map(t -> "ui::gridlayout::" + t + "::value")
.orElse(it.getKey()), it.getValue())))
.orElse(it.getKey()),
it.getValue())))
.collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
}

Expand Down
Loading
Loading