Retire legacy ImporterAPI bulk-transaction endpoint

.github/workflows/build_container.yml

@@ -209,7 +209,6 @@ jobs:
           echo tests.port=8016 >> obp-api/src/main/resources/props/test.default.props
           echo End of minimum settings >> obp-api/src/main/resources/props/test.default.props
           echo payments_enabled=false >> obp-api/src/main/resources/props/test.default.props
-          echo importer_secret=change_me >> obp-api/src/main/resources/props/test.default.props
           echo messageQueue.updateBankAccountsTransaction=false >> obp-api/src/main/resources/props/test.default.props
           echo messageQueue.createBankAccounts=false >> obp-api/src/main/resources/props/test.default.props
           echo allow_sandbox_account_creation=true >> obp-api/src/main/resources/props/test.default.props

.github/workflows/build_pull_request.yml

@@ -212,7 +212,6 @@ jobs:
           echo tests.port=8016 >> obp-api/src/main/resources/props/test.default.props
           echo End of minimum settings >> obp-api/src/main/resources/props/test.default.props
           echo payments_enabled=false >> obp-api/src/main/resources/props/test.default.props
-          echo importer_secret=change_me >> obp-api/src/main/resources/props/test.default.props
           echo messageQueue.updateBankAccountsTransaction=false >> obp-api/src/main/resources/props/test.default.props
           echo messageQueue.createBankAccounts=false >> obp-api/src/main/resources/props/test.default.props
           echo allow_sandbox_account_creation=true >> obp-api/src/main/resources/props/test.default.props

LIFT_HTTP4S_MIGRATION.md

@@ -225,7 +225,7 @@ Already partly described in the next major section, but counted here for complet
 | Endpoint | File | Notes |
 |---|---|---|
 | `aliveCheck` | `code/api/aliveCheck.scala` → `code/api/AliveCheckRoutes.scala` | **Done.** Native http4s route serves `GET /alive`; `LiftRules.statelessDispatch.append(aliveCheck)` removed from `Boot.scala`. |
-| `ImporterAPI` | `code/management/ImporterAPI.scala` → `code/management/ImporterAPIRoutes.scala` | **Done.** Native http4s route serves `POST /obp_transactions_saver/api/transactions`; secret read from URL query, body parsed from `req.bodyText`, `TransactionInserter` LiftActor still invoked synchronously (wrapped in `IO.blocking`). `ImporterTest` (8 scenarios) green. |
+| `ImporterAPI` | (deleted) | **Retired.** The legacy `POST /obp_transactions_saver/api/transactions` shared-secret bulk-insert endpoint, its `TransactionInserter` LiftActor, and the connector helpers it relied on (`createImportedTransaction`, `getMatchingTransactionCount`, `updateAccountBalance`, `setBankAccountLastUpdated`) have been removed entirely. Modern callers use connector-driven flows or the `/obp/vX.X.X/transaction-requests/...` endpoints. |
 | `OpenIdConnect` | (auth-stack table above) | OIDC callback, registered separately from OAuth2. |
 
 ### Open-banking standards (large, deferred indefinitely)
@@ -254,7 +254,7 @@ Three forks for how this workstream resolves:
 
 Currently runs on startup and goes away once the Lift bridge is removable:
 
-1. `LiftRules.statelessDispatch.append(...)` registrations: `DirectLogin`, `ImporterAPI`, `ResourceDocs140`–`ResourceDocs600`, `aliveCheck`.
+1. `LiftRules.statelessDispatch.append(...)` registrations: `DirectLogin`, `ResourceDocs140`–`ResourceDocs600`, `aliveCheck`.
 2. `LiftRules.dispatch.append(OpenIdConnect)`.
 3. `LiftRules.addToPackages("code")` — Lift package scanner.
 4. `LiftRules.exceptionHandler.prepend { ... }` — global exception handler.
@@ -297,7 +297,7 @@ A second decision is *not* required for bridge removal, but is required for the
 
 1. ~~**v4.0.0 bulk port**~~ — done (258/258, 100%).
 2. ~~**DirectLogin**~~ — done. `code.api.DirectLoginRoutes` serves the bare `/my/logins/direct`; per-version paths served by their own `Http4sXxx`. `LiftRules.statelessDispatch.append(DirectLogin)` retired.
-3. ~~**`aliveCheck`, `ImporterAPI`**~~ — done. `code.api.AliveCheckRoutes` serves `GET /alive`; `code.management.ImporterAPIRoutes` serves `POST /obp_transactions_saver/api/transactions`. Both Lift dispatches retired.
+3. ~~**`aliveCheck`**~~ — done. `code.api.AliveCheckRoutes` serves `GET /alive`; Lift dispatch retired. **`ImporterAPI`** — retired entirely (no http4s replacement); the legacy shared-secret bulk-transaction-importer endpoint has been removed along with `TransactionInserter` and the connector helpers it relied on.
 4. ~~**`Http4sResourceDocs` centralised service**~~ — done. `code.api.util.http4s.Http4sResourceDocs` serves `/obp/*/resource-docs/{API_VERSION}/{obp,swagger,openapi,openapi.yaml}`, `/obp/*/banks/{BANK_ID}/resource-docs/{API_VERSION}/obp`, and `/obp/*/message-docs/{CONNECTOR}/swagger2.0`. 10 `LiftRules.statelessDispatch.append(ResourceDocs140..600)` retired + `openapi.yaml` raw `serve { ... }` block removed. ResourceDocsTest (63) + SwaggerDocsTest (10) green.
 5. **Auth stack: OAuth2 / GatewayLogin / DAuth** — done. All three turned out to be library-only token validators (no `serve` blocks, no `LiftRules` registration). Vestigial `extends RestHelper` mixins removed.
 6. **OpenIdConnect** — the only remaining auth-stack work. Blocked on a portal-session decision (its success path calls `AuthUser.logUserIn` / `S.redirectTo`, which mutate Lift `SessionVar`s — see auth-stack table). OAuth 1.0a was removed entirely in commit `51820c75e`; no migration needed.

obp-api/src/main/resources/props/sample.props.template

@@ -463,9 +463,6 @@ connection.host=localhost
 connection.user=theusername
 connection.password=thepassword
 
-## Secret key that allows access to the "add transactions" api. You should change this to your own secret key
-importer_secret=change_me
-
 ## Set this to true if you want to have the api send a message to the hbci project to refresh transactions for an account
 messageQueue.updateBankAccountsTransaction=false
 

obp-api/src/main/resources/props/test.default.props.template

@@ -96,9 +96,6 @@ payments_enabled=false
 #connection.user=user
 #connection.password=pw
 
-#secret key that allows access to the "add transactions" api. You should change this to your own secret key
-importer_secret=change_me
-
 #set this to true if you want to have the api to send a message to the hbci project to refresh transactions for an account
 messageQueue.updateBankAccountsTransaction=false
 

obp-api/src/main/scala/bootstrap/liftweb/Boot.scala

@@ -498,13 +498,11 @@ class Boot extends MdcLoggable {
         LiftRules.dispatch.append(OpenIdConnect)
       }
     }
-    // DirectLogin (POST /my/logins/direct), ImporterAPI (POST
-    // /obp_transactions_saver/api/transactions), and aliveCheck (GET /alive)
-    // are now served by their native http4s counterparts wired into
-    // Http4sApp.baseServices (DirectLoginRoutes / ImporterAPIRoutes /
-    // AliveCheckRoutes). The Lift dispatches were retired in the http4s
-    // migration; any prop gates (e.g. `allow_direct_login`) live with those
-    // routes.
+    // DirectLogin (POST /my/logins/direct) and aliveCheck (GET /alive) are now
+    // served by their native http4s counterparts wired into
+    // Http4sApp.baseServices (DirectLoginRoutes / AliveCheckRoutes). The Lift
+    // dispatches were retired in the http4s migration; any prop gates
+    // (e.g. `allow_direct_login`) live with those routes.
 
 
 

obp-api/src/main/scala/code/api/util/http4s/Http4sApp.scala

@@ -123,7 +123,6 @@ object Http4sApp {
         .orElse(v121Routes.run(req))
         .orElse(code.api.DirectLoginRoutes.routes.run(req))
         .orElse(code.api.AliveCheckRoutes.routes.run(req))
-        .orElse(code.management.ImporterAPIRoutes.routes.run(req))
         .orElse(Http4sLiftWebBridge.routes.run(req))
     }
   }

obp-api/src/main/scala/code/bankconnectors/LocalMappedConnectorInternal.scala

@@ -19,7 +19,6 @@ import code.bankconnectors.ethereum.DecodeRawTx
 import code.branches.MappedBranch
 import code.fx.fx
 import code.fx.fx.TTL
-import code.management.ImporterAPI.ImporterTransaction
 import code.model.dataAccess.{BankAccountRouting, MappedBank, MappedBankAccount}
 import code.model.toBankAccountExtended
 import code.transaction.MappedTransaction
@@ -322,120 +321,6 @@ object LocalMappedConnectorInternal extends MdcLoggable {
   }
 
 
-  //transaction import api uses bank national identifiers to uniquely indentify banks,
-  //which is unfortunate as theoretically the national identifier is unique to a bank within
-  //one country
-  private def getBankByNationalIdentifier(nationalIdentifier: String): Box[Bank] = {
-    MappedBank.find(By(MappedBank.national_identifier, nationalIdentifier))
-  }
-
-  private def getAccountByNumber(bankId: BankId, number: String): Box[BankAccount] = {
-    MappedBankAccount.find(
-      By(MappedBankAccount.bank, bankId.value),
-      By(MappedBankAccount.accountNumber, number))
-  }
-
-  private val bigDecimalFailureHandler: PartialFunction[Throwable, Unit] = {
-    case ex: NumberFormatException => {
-      logger.warn(s"could not convert amount to a BigDecimal: $ex")
-    }
-  }
-
-  //used by transaction import api call to check for duplicates
-  def getMatchingTransactionCount(bankNationalIdentifier: String, accountNumber: String, amount: String, completed: Date, otherAccountHolder: String): Box[Int] = {
-    //we need to convert from the legacy bankNationalIdentifier to BankId, and from the legacy accountNumber to AccountId
-    val count = for {
-      bankId <- getBankByNationalIdentifier(bankNationalIdentifier).map(_.bankId)
-      account <- getAccountByNumber(bankId, accountNumber)
-      amountAsBigDecimal <- tryo(bigDecimalFailureHandler)(BigDecimal(amount))
-    } yield {
-
-      val amountInSmallestCurrencyUnits =
-        Helper.convertToSmallestCurrencyUnits(amountAsBigDecimal, account.currency)
-
-      MappedTransaction.count(
-        By(MappedTransaction.bank, bankId.value),
-        By(MappedTransaction.account, account.accountId.value),
-        By(MappedTransaction.amount, amountInSmallestCurrencyUnits),
-        By(MappedTransaction.tFinishDate, completed),
-        By(MappedTransaction.counterpartyAccountHolder, otherAccountHolder))
-    }
-
-    //icky
-    Full(count.map(_.toInt) getOrElse 0)
-  }
-
-
-  def createImportedTransaction(transaction: ImporterTransaction): Box[Transaction] = {
-    //we need to convert from the legacy bankNationalIdentifier to BankId, and from the legacy accountNumber to AccountId
-    val obpTransaction = transaction.obp_transaction
-    val thisAccount = obpTransaction.this_account
-    val nationalIdentifier = thisAccount.bank.national_identifier
-    val accountNumber = thisAccount.number
-    for {
-      bank <- getBankByNationalIdentifier(transaction.obp_transaction.this_account.bank.national_identifier) ?~!
-        s"No bank found with national identifier $nationalIdentifier"
-      bankId = bank.bankId
-      account <- getAccountByNumber(bankId, accountNumber)
-      details = obpTransaction.details
-      amountAsBigDecimal <- tryo(bigDecimalFailureHandler)(BigDecimal(details.value.amount))
-      newBalanceAsBigDecimal <- tryo(bigDecimalFailureHandler)(BigDecimal(details.new_balance.amount))
-      amountInSmallestCurrencyUnits = Helper.convertToSmallestCurrencyUnits(amountAsBigDecimal, account.currency)
-      newBalanceInSmallestCurrencyUnits = Helper.convertToSmallestCurrencyUnits(newBalanceAsBigDecimal, account.currency)
-      otherAccount = obpTransaction.other_account
-      mappedTransaction = MappedTransaction.create
-        .bank(bankId.value)
-        .account(account.accountId.value)
-        .transactionType(details.kind)
-        .amount(amountInSmallestCurrencyUnits)
-        .newAccountBalance(newBalanceInSmallestCurrencyUnits)
-        .currency(account.currency)
-        .tStartDate(details.posted.`$dt`)
-        .tFinishDate(details.completed.`$dt`)
-        .description(details.label)
-        .counterpartyAccountNumber(otherAccount.number)
-        .counterpartyAccountHolder(otherAccount.holder)
-        .counterpartyAccountKind(otherAccount.kind)
-        .counterpartyNationalId(otherAccount.bank.national_identifier)
-        .counterpartyBankName(otherAccount.bank.name)
-        .counterpartyIban(otherAccount.bank.IBAN)
-        .saveMe()
-      transaction <- mappedTransaction.toTransaction(account)
-    } yield transaction
-  }
-
-  //used by the transaction import api
-  def updateAccountBalance(bankId: BankId, accountId: AccountId, newBalance: BigDecimal): Box[Boolean] = {
-    //this will be Full(true) if everything went well
-    val result = for {
-      (bank, _) <- Connector.connector.vend.getBankLegacy(bankId, None)
-      account <- Connector.connector.vend.getBankAccountLegacy(bankId, accountId, None).map(_._1).map(_.asInstanceOf[MappedBankAccount])
-    } yield {
-      account.accountBalance(Helper.convertToSmallestCurrencyUnits(newBalance, account.currency)).save
-      setBankAccountLastUpdated(bank.nationalIdentifier, account.number, now).openOrThrowException(attemptedToOpenAnEmptyBox)
-    }
-
-    Full(result.getOrElse(false))
-  }
-
-  def setBankAccountLastUpdated(bankNationalIdentifier: String, accountNumber: String, updateDate: Date): Box[Boolean] = {
-    val result = for {
-      bankId <- getBankByNationalIdentifier(bankNationalIdentifier).map(_.bankId)
-      account <- getAccountByNumber(bankId, accountNumber)
-    } yield {
-      val acc = MappedBankAccount.find(
-        By(MappedBankAccount.bank, bankId.value),
-        By(MappedBankAccount.theAccountId, account.accountId.value)
-      )
-      acc match {
-        case Full(a) => a.accountLastUpdate(updateDate).save
-        case _ => logger.warn("can't set bank account.lastUpdated because the account was not found"); false
-      }
-    }
-    Full(result.getOrElse(false))
-  }
-
-
   //creates a bank account for an existing bank, with the appropriate values set. Can fail if the bank doesn't exist
   def createSandboxBankAccount(
     bankId: BankId,