Feature/symfony expression transformer

doc/03_Configuration/06_Mapping_Configuration/02_Transformation_Pipeline.md

@@ -68,6 +68,7 @@ and the second item reflects the value for the longitude attribute.
 - **Reduce Array KeyValue Pairs**: Reduces a flat array `['k1', 'v1', 'k2', 'v2']` 
    to key-value pair array `['k1' => 'v1', 'k2' => 'v2']`. 
 - **Trim**: Removes leading and/or tailing white spaces from string.
+- **Symfony Expression**: Evaluates a Symfony Expression Language expression against one or more source column values. Input values are available as attributes[0], attributes[1], etc., matching the order of source columns selected above in the pipeline. Returns the result of the expression. Custom functions can be registered via expression function providers — see Adding Expression Function Providers.
 - **Static Text**: Adds a static text to the value - either prepends or appends it.
 - **Conditional Conversion**: String values are converted to other string values (e.g. '0' to '1' or 'csv-value' to 'object-value'). Multiple conversions can be configured by separating the values with pipe symbol ('|') (e.g. '0|1|2' to 'some|other|values'). An asterisk can be used as a wildcard (e.g. '0|\*' to 'no value|default' where '0' will be converted to 'no value' and all other values to 'default').
 - **ObjectField**: Extracts the value for a specified field from a DataObject. This is similar to `ObjectFieldGetter` or `AnyGetter`.

doc/03_Configuration/06_Mapping_Configuration/04_Operators/06_Symfony_Expression_Providers.md

@@ -0,0 +1,73 @@
+# Adding Expression Function Providers
+
+The **Symfony Expression** operator in the data importer transformation pipeline
+supports custom functions via the Symfony Expression Language provider mechanism.
+This allows you to register reusable functions (e.g. `concat`, `replace`, `contains`)
+that can be called inside any expression configured in the operator.
+
+## How it works
+
+The operator evaluates expressions using `Pimcore\Bundle\DataImporterBundle\Utils\ExpressionLanguage\DataImporterExpressionLanguage`,
+which is a thin wrapper around Symfony's `ExpressionLanguage`. On boot, the container
+collects all services tagged with `pimcore.datahub.data_importer.expression_language_provider`
+and registers their functions automatically.
+
+## Creating a provider
+
+Implement `Symfony\Component\ExpressionLanguage\ExpressionFunctionProviderInterface`:
+
+```php
+<?php
+
+declare(strict_types=1);
+
+namespace App\ExpressionLanguage;
+
+use Symfony\Component\ExpressionLanguage\ExpressionFunction;
+use Symfony\Component\ExpressionLanguage\ExpressionFunctionProviderInterface;
+
+class MyFunctionsProvider implements ExpressionFunctionProviderInterface
+{
+    public function getFunctions(): array
+    {
+        return [
+            new ExpressionFunction(
+                'myFunction',
+                // Compiler (for compiled expressions — can throw if not needed)
+                function ($arg) {
+                    return sprintf('strtolower(%s)', $arg);
+                },
+                // Evaluator (used at runtime)
+                function (array $variables, $value): string {
+                    return strtolower((string) $value);
+                }
+            ),
+        ];
+    }
+}
+```
+
+Each `ExpressionFunction` takes three arguments:
+- **name** — the function name as used in expressions
+- **compiler** — a callable that returns a PHP expression string (used when expressions are compiled to PHP)
+- **evaluator** — a callable `(array $variables, ...$args)` that performs the actual work at runtime
+
+## Registering the provider
+
+Tag the service in `config/services.yaml`:
+
+```yaml
+App\ExpressionLanguage\MyFunctionsProvider:
+    tags:
+        - { name: 'pimcore.datahub.data_importer.expression_language_provider' }
+```
+
+## Using functions in an expression
+
+Once registered, the function name is available directly in any Symfony Expression
+operator. Input column values are exposed as `attributes[0]`, `attributes[1]`, etc.
+
+```
+myFunction(attributes[0])
+```
+

src/Mapping/Operator/Simple/SymfonyExpression.php

@@ -0,0 +1,50 @@
+<?php
+
+/**
+ * This source file is available under the terms of the
+ * Pimcore Open Core License (POCL)
+ * Full copyright and license information is available in
+ * LICENSE.md which is distributed with this source code.
+ *
+ *  @copyright  Copyright (c) Pimcore GmbH (https://www.pimcore.com)
+ *  @license    Pimcore Open Core License (POCL)
+ */
+
+namespace Pimcore\Bundle\DataImporterBundle\Mapping\Operator\Simple;
+
+use Pimcore\Bundle\DataImporterBundle\Utils\ExpressionLanguage\DataImporterExpressionLanguage;
+use Pimcore\Bundle\DataImporterBundle\Mapping\Operator\AbstractOperator;
+use Pimcore\Bundle\DataImporterBundle\Mapping\Type\TransformationDataTypeService;
+use Symfony\Contracts\Service\Attribute\Required;
+
+class SymfonyExpression extends AbstractOperator
+{
+    protected string $expression = '';
+
+    private DataImporterExpressionLanguage $expressionLanguage;
+
+    #[Required]
+    public function setExpressionLanguage(DataImporterExpressionLanguage $expressionLanguage): void
+    {
+        $this->expressionLanguage = $expressionLanguage;
+    }
+
+    public function setSettings(array $settings): void
+    {
+        $this->expression = trim($settings['expression'] ?? '');
+    }
+
+    public function process($inputData, bool $dryRun = false): mixed
+    {
+        if (empty($this->expression)) {
+            return $inputData;
+        }
+
+        return $this->expressionLanguage->evaluate($this->expression, ['attributes' => $inputData]);
+    }
+
+    public function evaluateReturnType(string $inputType, ?int $index = null): string
+    {
+        return TransformationDataTypeService::DEFAULT_TYPE;
+    }
+}

src/PimcoreDataImporterBundle.php

@@ -45,6 +45,99 @@
         return new PimcoreDataImporterExtension();
     }
 
+    /**
+     * @return string[]
+     */
+    public function getCssPaths(): array
+    {
+        return [
+            '/bundles/pimcoredataimporter/css/icons.css'
+        ];
+    }
+
+    /**
+     * @return string[]
+     */
+    public function getJsPaths(): array
+    {
+        return [
+            '/bundles/pimcoredataimporter/js/pimcore/helper/ext_extensions.js',
+            '/bundles/pimcoredataimporter/js/pimcore/helper/abstractOptionType.js',
+            '/bundles/pimcoredataimporter/js/pimcore/adapter/dataImporterDataObject.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/configEvents.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/configItemDataObject.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/loader/sftp.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/loader/http.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/loader/asset.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/loader/upload.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/loader/push.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/loader/sql.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/interpreter/csv.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/interpreter/json.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/interpreter/xlsx.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/interpreter/xml.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/interpreter/sql.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/cleanup/unpublish.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/cleanup/delete.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/importSettings.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/importPreview.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/resolver/load/id.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/resolver/load/path.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/resolver/load/attribute.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/resolver/load/notLoad.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/resolver/location/staticPath.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/resolver/location/findParent.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/resolver/location/findOrCreateFolder.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/resolver/location/noChange.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/resolver/location/doNotCreate.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/resolver/publish/alwaysPublish.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/resolver/publish/attributeBased.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/resolver/publish/noChangePublishNew.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/resolver/publish/noChangeUnpublishNew.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/mappingConfiguration.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/mappingConfigurationItem.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/transformationResultHandler.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/datatarget/direct.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/datatarget/manyToManyRelation.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/datatarget/classificationstore.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/datatarget/classificationstoreBatch.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/tools/classificationStoreKeySearchWindow.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/abstractOperator.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/trim.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/numeric.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/asArray.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/asCountries.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/asGeopoint.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/asGeobounds.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/asGeopolygon.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/asGeopolyline.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/asColor.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/explode.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/combine.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/htmlDecode.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/quantityValue.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/quantityValueArray.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/inputQuantityValue.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/inputQuantityValueArray.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/boolean.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/date.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/importAsset.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/loadAsset.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/gallery.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/imageAdvanced.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/loadDataObject.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/objectField.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/reduceArrayKeyValuePairs.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/flattenArray.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/staticText.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/conditionalConversion.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/stringReplace.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/mapping/operator/symfonyExpression.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/execution.js',
+            '/bundles/pimcoredataimporter/js/pimcore/configuration/components/logTab.js',
+        ];
+    }
+
     public function build(ContainerBuilder $container): void
     {
         $container

src/Resources/config/services.yml

@@ -192,3 +192,9 @@ services:
     Pimcore\Bundle\DataImporterBundle\Messenger\DataImporterHandler:
         tags:
             - { name: messenger.message_handler }
+
+    pimcore.datahub.data_importer.expression_language_provider:
+        class: Pimcore\Bundle\DataImporterBundle\Utils\ExpressionLanguage\DataImporterExpressionLanguage
+        public: true
+        arguments:
+            - !tagged_iterator pimcore.datahub.data_importer.expression_language_provider

src/Resources/config/services/mapping.yml

@@ -66,6 +66,10 @@ services:
         tags:
             - { name: "pimcore.datahub.data_importer.operator", type: "objectField" }
 
+    Pimcore\Bundle\DataImporterBundle\Mapping\Operator\Simple\SymfonyExpression:
+        tags:
+            - { name: "pimcore.datahub.data_importer.operator", type: "symfonyExpression" }
+
     # -------------------
     # factory operators
     # -------------------

src/Resources/public/js/pimcore/configuration/components/mapping/operator/symfonyExpression.js

@@ -0,0 +1,68 @@
+/**
+* This source file is available under the terms of the
+* Pimcore Open Core License (POCL)
+* Full copyright and license information is available in
+* LICENSE.md which is distributed with this source code.
+*
+*  @copyright  Copyright (c) Pimcore GmbH (http://www.pimcore.com)
+*  @license    Pimcore Open Core License (POCL)
+*/
+pimcore.registerNS("pimcore.plugin.pimcoreDataImporterBundle.configuration.components.mapping.operator.symfonyExpression");
+pimcore.plugin.pimcoreDataImporterBundle.configuration.components.mapping.operator.symfonyExpression = Class.create(pimcore.plugin.pimcoreDataImporterBundle.configuration.components.mapping.abstractOperator, {
+        type: "symfonyExpression",
+
+        getMenuGroup: function () {
+            return this.menuGroups.dataManipulation;
+        },
+
+        buildTransformationPipelineItem: function () {
+            const id = Ext.id();
+            if (!this.form) {
+                this.form = Ext.create("DataHub.DataImporter.StructuredValueForm", {
+                    operatorImplementation: this,
+                    id: id,
+                    style: "margin-top: 10px",
+                    border: true,
+                    bodyStyle: "padding: 10px;",
+                    tbar: this.getTopBar(t("plugin_pimcore_datahub_data_importer_configpanel_transformation_pipeline_symfonyExpression"), id, this.container),
+                    items: this.getFormItems(),
+                });
+            }
+            return this.form;
+        },
+
+        getFormItems: function () {
+            const savedExpression =
+                this.data.settings && this.data.settings.expression
+                    ? this.data.settings.expression
+                    : "";
+
+            const infoHtml =
+                '<div style="font-size:11px; padding:6px 8px; background:#f0f4f8; border:1px solid #c8d6e5; border-radius:3px; margin-bottom:6px;">' +
+                '<strong style="display:block; margin-bottom:4px;">Available variables:</strong>' +
+                '<div style="font-family:monospace; margin-bottom:4px;">attributes[0], attributes[1], &hellip;</div>' +
+                '<div style="color:#888; font-style:italic;">Values come from the source columns selected above in the pipeline.</div>' +
+                "</div>";
+
+            return [
+                {
+                    xtype: "box",
+                    html: infoHtml,
+                },
+                {
+                    xtype: "textarea",
+                    fieldLabel: t("plugin_pimcore_datahub_data_importer_configpanel_transformation_pipeline_symfonyExpression"),
+                    labelAlign: "top",
+                    name: "settings.expression",
+                    value: savedExpression,
+                    height: 100,
+                    anchor: "100%",
+                    emptyText: "e.g. attributes[0] == 'Demo Value' ? attributes[1] : null",
+                    listeners: {
+                        change: this.inputChangePreviewUpdate.bind(this),
+                    },
+                },
+            ];
+        },
+    }
+);