Integrate LOBSTER apps into crystal toolkit

[JaGeo]

This is the PR to integrate LOBSTER into the Crystal Toolkit.

See below for a detailed explanation of the PR by @naik-aakash !

[naik-aakash]

StructureMoleculeComponent update

• @JaGeo added option to use LobsterNeighbors in structure visualization

CohpAndDosComponent Features

• Get relevant bonds interactive cohp plots alongside dos from Lobstery by means of LobsterPy and existing dos plotter in crystaltoolkit
• Include Bonding analysis summary and calculation quality texts
• Include visualization of LobsterEnv identified environments
• Hovering over atoms shows Mulliken charges and edges show the ICOHP values in the visualizer

Example rendered app

Todos (Will be worked on once app layout is finalized))

• Exception handling needs to be implemented
• Add tests

Open questions and things to improve

• App loading is currently slow as we pass in completecohp objects (this is quite large) as due to inherent nature of how ploty/dash handles callbacks, lot of time is spent in serialization and deserialization of this object

• Vasprun.xml file seems to have issues with serialization via plotly so, I think, simple fix would be to just make a small change in pymatgen - Have included temporary workaround for it in the example script here.

• Icohpcollection object also has serialization issues and currently a fix is a WIP via this Add custom as_dict/from_dict method for proper initialization of attributes of IcohpCollection pymatgen#4391

• With the current changes LobsterNeighbors can also be used to visualize structure on the Main MP website. Is this desirable ? We think could be nice addition to already exisitng options.
  

• Need to agree on what would be best place to visualize LocalEnv identified from LobsterNeighbors, maybe in crystaltoolkit app ? (i.e move this part in localenv.py module ) or could be better to have all LOBSTER related information in one page as in current app layout?

• What one can query over from MP website using mpid ? If we intend to provide complete taskdoc then we can then add option as in the app.

[esoteric-ephemera]

@naik-aakash @JaGeo for this to be used on MP's site, we'll need any of the required outputs to be in the LobsterTaskDocument. The client will retrieve a LobsterTaskDocument and then use any fields, like CalcQualitySummary rather than rerunning Analysis.get_lobster_calc_quality_summary

(The client shouldn't depend on lobsterpy / need to call lobsterpy). Also to be clear, crystal toolkit shouldn't depend on emmet, but should use data that is available in the task document

The method you implemented is OK to keep as a from_files / from_directory method, but we'd also need something that operates on data after lobster is run

Can you double check that any data that is required for the app to work is included in the task document model? Any updates to the schema should be made against this branch in emmet (PR link)

[naik-aakash]

Hi @esoteric-ephemera, thanks for the suggestion. I will look at the pinned emmet PR and raise a PR if I find anything is missing.

I will also try to make the changes in the client in the PR soon.

[esoteric-ephemera]

For this class, I would recommend against copying all of Vasprun.final_energy:

class CustomVasprun(Vasprun):

    @property
    def final_energy(self):
        return float(super().final_energy)

[naik-aakash]

Hi @esoteric-ephemera , have one question. So, now the plan is that LobsterPy is already run as part of taskdoc generation, and we simply use the outputs from the taskdoc and render the app?

I did it this way because, in our initial discussions, it was emphasized that Postprocessing should not be part of Taskdoc, but rather that this app should handle it.

Curious about how the workflow pipeline would be now.

Once I start with this, I can better comment on the emmet PR.

[esoteric-ephemera]

Any "heavy" post-processing, or post-processing which requires raw output files like vasprun.xml, should happen in the build process for your data. In the current pipeline, the lobster build would be LobsterTaskDocument.from_directory

Ex: all derived MP data is built from TaskDoc objects, and not the raw VASP output - any necessary fields for apps/subsequent builds has to be in TaskDoc. The tasks collection is built from the TaskDoc.from_directory method currently

Light post-processing can occur in the app (e.g., the oxygen evolution graph drawing in the client) , but this should be

Let me know if that makes sense!

[naik-aakash]

Hi @esoteric-ephemera , I made the changes are requested to directly work with taskdoc inputs. Made some tweaks and added some more options to enable more interactive anaylsis.

Here is screenshot of updated layout

Also I have an additional code that can create such visualization using plotly and dash. Have attached a short video of it here

Screencast.from.06.02.2026.17.03.33.webm

I could not figure out to reuse the Structuremolecule component for this, as I wanted clickable edges. Not sure if we like to have such thing on materials project website. Also as it relies on cohpcar objects, not sure if this can easily work on website. So have not pushed the code here yet.

Let me know what you think? Also if this additional app is desired, It would be great if anyone can help with modifying Structuremolecule component. If it is too much I can simply include it in LobsterPy and we can provide a button to tell users to make use of LobsterPy for more powerful interactive analysis.

[esoteric-ephemera]

Thanks, @naik-aakash! Looking good and perhaps @minhsueh has suggestions on how to modify / work with existing components

We can discuss this more, but would this ideally be included as a separate property tab for a material, or in a standalone app?

[naik-aakash]

Hi @minhsueh, @JaGeo I have added the test and addressed the review comments that I could. Am confused why the added test is failing here, but it passes locally without any errors.

[minhsueh]

@naik-aakash
For the failing test, I haven't been able to reproduce it locally because our environments differ. However, based on the error message:

FAILED crystal_toolkit/apps/examples/tests/test_lobsterenv.py::test_localenv_example_renders_lobsterenv_controls - selenium.common.exceptions.TimeoutException: Message: timeout 30s => waiting for selector [id$='perc_strength_icohp']

My guess is that the LOBSTER analysis is taking longer than expected, causing the page to exceed the 30-second timeout.

One suggestion before directly increasing the timeout: if you're able to reproduce the issue locally, try running the test in headed mode with --slowmo enabled. That should make it easier to see whether the page is simply loading slowly or if another issue appears before the timeout.

python -m pytest --cov=crystal_toolkit --cov-report=xml --color=yes --webdriver Chrome --headed --slowmo 3000 crystal_toolkit/apps/examples/tests/test_lobsterenv.py

[naik-aakash]

@naik-aakash For the failing test, I haven't been able to reproduce it locally because our environments differ. However, based on the error message:

FAILED crystal_toolkit/apps/examples/tests/test_lobsterenv.py::test_localenv_example_renders_lobsterenv_controls - selenium.common.exceptions.TimeoutException: Message: timeout 30s => waiting for selector [id$='perc_strength_icohp']

My guess is that the LOBSTER analysis is taking longer than expected, causing the page to exceed the 30-second timeout.

One suggestion before directly increasing the timeout: if you're able to reproduce the issue locally, try running the test in headed mode with --slowmo enabled. That should make it easier to see whether the page is simply loading slowly or if another issue appears before the timeout.

python -m pytest --cov=crystal_toolkit --cov-report=xml --color=yes --webdriver Chrome --headed --slowmo 3000 crystal_toolkit/apps/examples/tests/test_lobsterenv.py

Hi @minhsueh , I created a fresh new environment and installed the same dependencies as in the current CI workflow.

https://github.com/JaGeo/crystaltoolkit/blob/8ac49a781872cc385981116b479f2baba67d7613/.github/workflows/pull-request.yml#L38-L43

It still seems to pass without any issues. The page seemed to load properly and navigate to the expected dropdown option "LobsterEnv" before running the assertions. No errors I could see.

[naik-aakash]

@naik-aakash For the failing test, I haven't been able to reproduce it locally because our environments differ. However, based on the error message:

FAILED crystal_toolkit/apps/examples/tests/test_lobsterenv.py::test_localenv_example_renders_lobsterenv_controls - selenium.common.exceptions.TimeoutException: Message: timeout 30s => waiting for selector [id$='perc_strength_icohp']

My guess is that the LOBSTER analysis is taking longer than expected, causing the page to exceed the 30-second timeout.
One suggestion before directly increasing the timeout: if you're able to reproduce the issue locally, try running the test in headed mode with --slowmo enabled. That should make it easier to see whether the page is simply loading slowly or if another issue appears before the timeout.

python -m pytest --cov=crystal_toolkit --cov-report=xml --color=yes --webdriver Chrome --headed --slowmo 3000 crystal_toolkit/apps/examples/tests/test_lobsterenv.py

Hi @minhsueh , I created a fresh new environment and installed the same dependencies as in the current CI workflow.

https://github.com/JaGeo/crystaltoolkit/blob/8ac49a781872cc385981116b479f2baba67d7613/.github/workflows/pull-request.yml#L38-L43

It still seems to pass without any issues. The page seemed to load properly and navigate to the expected dropdown option "LobsterEnv" before running the assertions. No errors I could see.

Seems now the test passed, not sure if it failed yesterday use to issues of github actions yesterday or now it passed cause of my changes here..

[minhsueh]

@naik-aakash Thanks for the test fix. I reran the test and it failed twice (see Attempt # 2 and # 3), but the failure doesn't appear to be related to lobsterenv.

• Once you've updated _extract_structure_from_data
• And also, could you remove crystal_toolkit/_version.py? It's generated by setuptools_scm and shouldn't be committed.
• Merge with the latest main branch.

I'll merge this PR.

[naik-aakash]

@naik-aakash Thanks for the test fix. I reran the test and it failed twice (see Attempt # 2 and # 3), but the failure doesn't appear to be related to lobsterenv.

• Once you've updated _extract_structure_from_data
• And also, could you remove crystal_toolkit/_version.py? It's generated by setuptools_scm and shouldn't be committed.
• Merge with the latest main branch.

I'll merge this PR.

Hi @minhsueh , I have updated the _extract_structure_from_data and also removed accidentaly commited crystal_toolkit/_version.py file.

[minhsueh]

LGTM! Thanks! @naik-aakash @JaGeo

[JaGeo]

Congrats @naik-aakash and thanks @minhsueh !!