Conversation
maksaska
changed the title
…istry lookup
Metric registry MBeans were looked up with ad-hoc patterns at every call site,
and the node's JMX client was memoized for the whole life of the node object.
* metric_registry_pattern(registry) builds the exact MBean name pattern of a
registry: split at the first dot like JmxMetricExporterSpi, values spelled
like IgniteUtils.escapeObjectNameValue(), escaped for grep -E, anchored at
the end of the line ('name' sorts last). A registry without a group is
filtered from the system view of the same name. The old
'group=cacheGroups.*name="%s"' pattern could not find an alphanumeric
(unquoted) name at all. Names with a quote, a backslash or '?' are
rejected, as the jmxterm command quoting cannot carry them.
* The node's JmxClient is rebuilt on every start_node(). Before, a restarted
node kept the pid and the clsLdr-bearing MBean names of its previous JVM,
so callers had to build a client of their own. JmxClient takes an explicit
pid for the CDC JVM.
* node.metric_registry_mbean() and JmxMBean.value()/bool_value() replace the
hand-built patterns and next(mbean.X) reads in rebalance, dump, snapshot
and CDC.
check_jmx_utils.py runs every pattern case through Python re and grep -E.
maksaska
force-pushed
the
ignite-28952-v2
branch
from
September 17, 2026 12:03
4f3da1a to
95903ff
Compare
An unescaped '[' right after the class opener made Python emit 'FutureWarning: Possible nested set' on every JMX metric lookup.
…data centers
MdcTopologyValidator has two modes and the number of data centers picks one:
with an EVEN DC count a segment stays writable while it sees the main DC, with
an ODD one while it sees a majority of the DC set. The fixture only ever built a
two DC cluster, so only the first mode could be tested.
MdcCluster now takes the DC set it spans (dcs=, two by default) and compiles the
cache parameters from it:
* mdc_topology_params() emits mainDc or datacenters, never both -
MdcTopologyValidator.checkConfiguration() rejects the pair;
* min_backups() is the smallest backup count giving every DC one copy of every
partition, and is what generate_data() uses by default;
* _with_cache_params() is the single point an application that creates the
cache is handed all of it, so no call site can configure a cache that
disagrees with the DC set. That is what lets the transactional test drop its
hand written mainDc.
verify_half_ring_healthy()/verify_split_brain() generalize into
verify_segment_healthy()/verify_segments(): a segment is now a DC or a group of
DCs that still see each other, so a three DC cluster with one DC cut off is
expressed as verify_segments((DC_1, DC_2), DC_3).
NetworkGroupManager gains enable/disable_network_partitions(*pairs). A cluster of
three or more groups is cut apart along several links at once, and every chain a
node takes part in has to be installed by the same single SSH round-trip: rolling
the links out one after the other would present the cluster with intermediate
segmentations it would legitimately react to.
The cache level MdcTopologyValidator becomes optional, through the cache
parameter topologyValidator and the mdc_cache_topology_validator global, for a
fork whose validator is configured elsewhere. The affinity backup filter moves
into a protected MdcCacheAwareApplication.backupFilter() for the same reason: a
fork that spreads the copies by something finer than the data center overrides
one method rather than repeating the cache configuration.
check_mdc_cluster.py and check_partition.py cover all of the above without a
cluster.
…ed validation
Three scenarios over a three DC cluster, each asserting something two DC mode
cannot show:
* test_minority_dc_isolation - one DC is cut off. The two DCs left keep
writing, the isolated one goes read-only while still serving every read.
Parametrized over the isolated DC, because no DC is privileged here: cutting
off DC1 - the main DC of the equivalent two DC cluster, and the DC the others
discovered through - is just as survivable as cutting off DC3.
* test_three_way_split_blocks_all_writes - every cross-DC link drops at once.
No segment holds a majority, so unlike the two DC case, where one half always
survives as writable, the whole cluster goes read-only.
* test_writes_survive_single_dc_loss - the same guarantee without any network
impairment: losing one DC leaves a majority, losing a second one does not.
Every partition owns exactly one copy per DC (backups = 2), so every segment
down to a single isolated DC still serves every read - which is what separates
the read assertions from the write ones throughout.
test_minority_dc_isolation also reads the cluster's own verdict on the MDC
guarantees off a node over JMX. IsCacheAffinityConfigurationMdcSafe is about the
cache CONFIGURATION and holds in every segment; IsCachePartitionDistributionSafe
is about the current assignment, and is only reported while the cluster is split
until the expected value is confirmed by a run.
…ric case
* cache_mdc_metrics() reads through node.metric_registry_mbean(): the node's
JMX client is restart-safe since IGNITE-29049, so the manual JmxClient
workaround is gone.
* The MDC safety metrics are also checked to say False: a cache created with
backupFilter=False (new MdcCacheAwareApplication parameter) is neither
affinity- nor distribution-safe. It is destroyed before the network cut,
since the isolated DC would lose its partitions. Adds
ControlUtility.cache_destroy().
* Docstrings of backupFilter(), _server_service() and dc_servers() reworded.
The unsafe cache check is not a functional scenario: the metric verdict for a cache without the MDC backup filter is already covered by MdcCacheMetricsTest. Creating and destroying an extra cache right before the network cut only prolongs the test and adds discovery load at the cut. Removes the backupFilter application parameter and ControlUtility.cache_destroy() along with it.
maksaska
force-pushed
the
ignite-28952-v2
branch
from
September 17, 2026 14:51
95903ff to
1904934
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thank you for submitting the pull request to the Apache Ignite.
In order to streamline the review of the contribution
we ask you to ensure the following steps have been taken:
The Contribution Checklist
The description explains WHAT and WHY was made instead of HOW.
The following pattern must be used:
IGNITE-XXXX Change summarywhereXXXX- number of JIRA issue.(see the Maintainers list)
the
green visaattached to the JIRA ticket (see tabPR Checkat TC.Bot - Instance 1 or TC.Bot - Instance 2)Notes
If you need any help, please email dev@ignite.apache.org or ask anу advice on http://asf.slack.com #ignite channel.