本体生成与校验

1"""2Regression tests for #1104 and #1105.34#1104: SHACLGenerator used one namespace for two different jobs. `base_uri`5names where the shape resources live, and it was also used to expand every6sh:targetClass and sh:path. With the default `https://semantica.dev/shapes/`7that made shapes target `.../shapes/Person`, while data carries8`.../ns#Person` or the ontology's own class IRI. The shapes matched nothing.9pySHACL then reported conforms=True, because a shape with no focus nodes is10vacuously satisfied, so the mismatch was invisible to the validator the11package ships with.1213#1105: a property with no declared domain was attached to every node shape,14which invents a constraint the ontology never stated. With minCount 1 that15makes every instance of every class invalid.1617These tests validate real data through pySHACL rather than reading the shapes18text, so a shape that targets nothing cannot pass by being ignored.19"""

1"""2Ontology Generation Module34This module handles automatic generation of ontologies from data and text using5a 6-stage pipeline that transforms raw data into structured OWL ontologies.67Key Features:8    - Automatic ontology generation (6-stage pipeline)9    - Class and property inference10    - Ontology structure optimization11    - Domain-specific ontology creation12    - Ontology quality assessment13    - Semantic network parsing14    - Hierarchy generation1516Main Classes:17    - OntologyGenerator: Main ontology generation class (6-stage pipeline)

49class OntologyGenerator:50    """51    Ontology generation handler with 6-stage pipeline.5253    6-Stage Pipeline:54    1. Semantic Network Parsing → Extract domain concepts55    2. YAML-to-Definition → Transform into class definitions56    3. Definition-to-Types → Map to OWL types57    4. Hierarchy Generation → Build taxonomic structures58    5. TTL Generation → Generate OWL/Turtle syntax using rdflib, triplet generation (subject-predicate-object)59    6. Symbolic Validation → HermiT/Pellet reasoning6061    • Generates ontologies from data and text62    • Infers classes and properties automatically63    • Creates domain-specific ontologies64    • Optimizes ontology structure65    • Validates ontology quality66    • Supports various ontology formats67    """

197            # Stage 6: Symbolic Validation198            if options.get("validate", True):199                self.progress_tracker.update_tracking(200                    tracking_id, message="Stage 6: Validating ontology..."201                )202                validation_result = self.validator.validate(ontology)203                ontology["validation"] = {204                    "valid": validation_result.valid,205                    "consistent": validation_result.consistent,206                    "satisfiable": validation_result.satisfiable,207                    "errors": validation_result.errors,208                    "warnings": validation_result.warnings209                }210                if not validation_result.valid:211                    self.logger.warning(f"Ontology validation failed: {validation_result.errors}")

226    def generate_from_graph(self, graph: Dict[str, Any], **options) -> Dict[str, Any]:227        """228        Generate ontology from a knowledge graph.229230        Alias for generate_ontology.231232        Args:233            graph: Knowledge graph dictionary (output from GraphBuilder)234            **options: Additional options235236        Returns:237            Generated ontology dictionary238        """239        return self.generate_ontology(graph, **options)

249        raw_entities = data.get("entities", [])250        relationships = data.get("relationships", [])251252        # Normalize entities253        entities = []254255        def process_entity(ent):256            """Normalize entity to dictionary."""257            if isinstance(ent, dict):258                return ent259            # Handle Entity object (from NERExtractor)260            if hasattr(ent, "label") and hasattr(ent, "text"):261                return {262                    "type": ent.label,263                    "name": ent.text,264                    "entity_type": ent.label,265                    "text": ent.text,266                    "start_char": getattr(ent, "start_char", 0),267                    "end_char": getattr(ent, "end_char", 0),268                    "confidence": getattr(ent, "confidence", 1.0),269                    "metadata": getattr(ent, "metadata", {}),270                }271            # Handle list/tuple (legacy or raw format)272            if isinstance(ent, (list, tuple)) and len(ent) >= 2:273                # Assume [text, label, ...]274                return {275                    "name": str(ent[0]),276                    "text": str(ent[0]),277                    "type": str(ent[1]),278                    "entity_type": str(ent[1]),279                }280            return None281282        # Handle list of lists (batch output) or flat list283        for item in raw_entities:284            if isinstance(item, list):285                # Check if it's a list of entities (batch) or a single entity as list286                # If the first element is also a list or Entity object, it's a batch287                if len(item) > 0 and (288                    isinstance(item[0], list) or hasattr(item[0], "label")289                ):290                    for sub_item in item:291                        processed = process_entity(sub_item)

151            # Infer classes from entity types152            self.progress_tracker.update_tracking(153                tracking_id,154                message=f"Inferring classes from {len(entity_types)} entity types...",155            )156            classes = []157            for entity_type, type_entities in entity_types.items():158                if len(type_entities) >= self.min_occurrences:159                    class_def = self._create_class_from_entities(160                        entity_type, type_entities, **options161                    )162                    classes.append(class_def)

308        # Simple heuristic: look for more general class names309        words = class_name.split()310311        # Try to find parent by removing words312        for i in range(len(words) - 1, 0, -1):313            parent_candidate = "".join(words[:i])314            if parent_candidate in class_map:315                return parent_candidate316317        # Check for common parent classes318        common_parents = ["Entity", "Thing", "Resource"]319        for parent in common_parents:320            if parent in class_map:321                return parent322323        return None

133    def _infer_object_properties(134        self,135        relationships: List[Dict[str, Any]],136        classes: List[Dict[str, Any]],137        **options,138    ) -> List[Dict[str, Any]]:139        """Infer object properties from relationships."""140        # Group relationships by type141        rel_types = defaultdict(list)142        for rel in relationships:143            rel_type = rel.get("type") or rel.get("relationship_type", "relatedTo")144            rel_types[rel_type].append(rel)145146        # Create class map147        class_map = {cls["name"]: cls for cls in classes}148149        properties = []150        for rel_type, rels in rel_types.items():151            if len(rels) >= options.get("min_occurrences", 2):152                # Infer domain and range153                domains = set()154                ranges = set()155156                for rel in rels:157                    source_type = rel.get(158                        "source_type"159                    ) or self._infer_class_from_entity(rel.get("source_id"), classes)160                    target_type = rel.get(161                        "target_type"162                    ) or self._infer_class_from_entity(rel.get("target_id"), classes)163164                    if source_type:165                        domains.add(source_type)166                    if target_type:167                        ranges.add(target_type)168169                # Normalize property name170                prop_name = self.naming_conventions.normalize_property_name(171                    rel_type, "object"172                )

269    def _infer_property_type(self, value: Any) -> str:270        """Infer property type from value."""271        if isinstance(value, bool):272            return "xsd:boolean"273        elif isinstance(value, int):274            return "xsd:integer"275        elif isinstance(value, float):276            return "xsd:double"277        elif isinstance(value, str):278            # Check if it's a date279            if self._is_date(value):280                return "xsd:date"281            elif self._is_datetime(value):282                return "xsd:dateTime"283            else:284                return "xsd:string"285        else:286            return "xsd:string"

443        # Add types to classes.444        # ClassInferrer sets "uri": None when it was given no namespace manager,445        # so the key is present and a `not in` guard never fires: every class446        # then reached the exporters with no IRI at all (#1103).447        for cls in classes:448            cls["@type"] = "owl:Class"449            if not cls.get("uri"):450                cls["uri"] = self.namespace_manager.generate_class_iri(cls["name"])451452        # Add types to properties453        for prop in properties:454            if prop["type"] == "object":455                prop["@type"] = "owl:ObjectProperty"456            else:457                prop["@type"] = "owl:DatatypeProperty"458459            if not prop.get("uri"):460                prop["uri"] = self.namespace_manager.generate_property_iri(prop["name"])

113        # Convert to PascalCase114        class_name = self._to_pascal_case(class_name)115116        # Generate speaking IRI if enabled117        if self.use_speaking_iris:118            iri = urljoin(self.get_base_uri(), class_name)119        else:120            # Use hash-based IRI121            import hashlib122123            hash_id = hashlib.md5(class_name.encode()).hexdigest()[:8]  # nosec B324 - deterministic IRI suffix, not security-sensitive124            iri = urljoin(self.get_base_uri(), f"class/{hash_id}")125126        return iri

215        gen_ns_manager = self._get_generation_namespace_manager(ontology)216217        # Set up namespaces218        ns_manager = RDFNamespaceManager(g)219220        # Register standard namespaces221        for prefix, uri in self.namespace_manager.get_all_namespaces().items():222            ns = Namespace(uri)223            ns_manager.bind(prefix, ns)224            g.bind(prefix, ns)225226        # Register ontology namespace227        base_uri = ontology.get("uri") or self.namespace_manager.get_base_uri()228        if isinstance(base_uri, str) and not base_uri.endswith(("/", "#")):229            base_uri = base_uri + "/"230        ont_ns = Namespace(base_uri)231        g.bind("", ont_ns)232233        # Create ontology resource234        ont_uri = URIRef(base_uri)235        g.add((ont_uri, RDF.type, OWL.Ontology))236237        # Add ontology metadata238        if ontology.get("name"):239            g.add((ont_uri, RDFS.label, Literal(ontology["name"])))240        if ontology.get("version"):241            g.add((ont_uri, OWL.versionInfo, Literal(ontology["version"])))242243        # Add classes244        classes = ontology.get("classes", [])245        for cls in classes:246            class_name = self._resolve_class_identifier(cls)247            class_uri = URIRef(248                cls.get("uri")249                or gen_ns_manager.generate_class_iri(class_name)250            )

794class SHACLGenerator:795    """796    Generates SHACL shapes from Semantica OWL ontology dicts.797798    6-stage internal pipeline:799        1. _build_class_index()       — {class_name: class_dict} for O(1) lookup800        2. _generate_node_shapes()    — one NodeShape per OWL class801        3. _attach_property_shapes()  — map properties to their domain node shapes802        4. _propagate_inheritance()   — copy parent shapes to children (iterative, cycle-safe)803        5. _apply_quality_tier()      — strict tier: set closed=True on all shapes804        6. serialize()                — Turtle / JSON-LD / N-Triples805    """

732@dataclass733class PropertyShape:734    """Internal model for a SHACL sh:PropertyShape."""735    path: str736    name: Optional[str] = None737    description: Optional[str] = None738    datatype: Optional[str] = None       # sh:datatype739    class_: Optional[str] = None         # sh:class740    min_count: Optional[int] = None741    max_count: Optional[int] = None742    in_values: Optional[List[str]] = None743    has_value: Optional[str] = None744    pattern: Optional[str] = None745    severity: str = "Violation"746747748@dataclass749class NodeShape:750    """Internal model for a SHACL sh:NodeShape."""751    target_class: str752    name: Optional[str] = None753    description: Optional[str] = None754    property_shapes: List[PropertyShape] = field(default_factory=list)755    closed: bool = False756    severity: str = "Violation"

778@dataclass779class SHACLGraph:780    """Internal model representing the complete SHACL shapes graph."""781    base_uri: str782    shapes_uri: str783    node_shapes: List[NodeShape] = field(default_factory=list)784    prefixes: Dict[str, str] = field(default_factory=dict)785    # Bare names mapped to the absolute IRI the data uses for them. Shapes are786    # indexed internally by name; this is what those names expand to at787    # serialisation time (#1104). Classes and properties are kept apart because788    # a property may legitimately share a class's name, and a single map would789    # silently give it the class's IRI.790    class_iris: Dict[str, str] = field(default_factory=dict)791    property_iris: Dict[str, str] = field(default_factory=dict)

887            target_ns = self._resolve_target_namespace(ontology, base_uri)888            prefixes["ex"] = target_ns889890            graph = SHACLGraph(891                base_uri=base_uri,892                shapes_uri=self.shapes_uri,893                prefixes=prefixes,894                class_iris=self._build_term_index(classes, target_ns),895                property_iris=self._build_term_index(properties, target_ns),896            )897898            self.progress_tracker.update_tracking(tracking_id, message="Generating node shapes")899            class_index = self._build_class_index(classes)900            self._generate_node_shapes(graph, classes)901902            self.progress_tracker.update_tracking(tracking_id, message="Attaching property shapes")903            self._attach_property_shapes(graph, properties)904905            if self.include_inherited:906                self.progress_tracker.update_tracking(tracking_id, message="Propagating inheritance")907                self._propagate_inheritance(graph, class_index)908909            self._apply_quality_tier(graph)

977    def _resolve_target_namespace(self, ontology: Dict[str, Any], base_uri: str) -> str:978        """979        Decide which namespace sh:targetClass and sh:path are expanded in.980981        base_uri says where the shapes live. It is only the right answer here982        when the ontology declared it, meaning shapes and terms deliberately983        share a namespace. Falling back to the shapes namespace produces shapes984        that target terms no data graph uses.985        """986        if self.target_namespace:987            return self.target_namespace988989        namespace = ontology.get("namespace")990        if isinstance(namespace, dict) and namespace.get("base_uri"):991            return str(namespace["base_uri"])992993        # An IRI already carried by a term is the most reliable evidence of994        # where the data lives, so prefer it over any configured default.995        for terms in (ontology.get("classes"), ontology.get("properties")):996            for term in terms or []:997                if not isinstance(term, dict):998                    continue999                for key in ("uri", "iri", "id"):1000                    value = term.get(key)1001                    if self._is_absolute_iri(value):1002                        value = value.strip()1003                        cut = max(value.rfind("#"), value.rfind("/"))1004                        if cut != -1:1005                            return value[: cut + 1]10061007        if ontology.get("uri") and self._is_absolute_iri(ontology["uri"]):1008            return str(ontology["uri"])10091010        if base_uri != self.base_uri:1011            return base_uri10121013        return self._DEFAULT_TARGET_NAMESPACE

1058    def _attach_property_shapes(1059        self, graph: SHACLGraph, properties: List[Dict[str, Any]]1060    ) -> None:1061        shape_by_class = {ns.target_class: ns for ns in graph.node_shapes}10621063        for prop in properties:1064            pname = prop.get("name")1065            if not pname:1066                continue10671068            domain = prop.get("domain")1069            if isinstance(domain, list):1070                domains = [d for d in domain if d]1071            elif isinstance(domain, str) and domain:1072                domains = [domain]1073            else:1074                domains = []10751076            if domains:1077                for d in domains:1078                    if d in shape_by_class:1079                        shape_by_class[d].property_shapes.append(1080                            self._build_property_shape(prop)1081                        )1082                    else:1083                        self.logger.debug(1084                            f"Property '{pname}' domain '{d}' has no matching node shape — skipped"1085                        )1086            elif self.attach_domainless_properties:1087                self.logger.warning(1088                    f"Property '{pname}' declares no domain and is being attached "1089                    "to every node shape because attach_domainless_properties is "1090                    "set. This states a constraint the ontology does not."1091                )1092                for node_shape in graph.node_shapes:1093                    node_shape.property_shapes.append(self._build_property_shape(prop))1094            else:1095                # Attaching here would state a constraint the ontology does not.1096                # With minCount 1 that invalidates every instance of every1097                # class, so the property is left unattached (#1105).1098                self.logger.warning(1099                    f"Property '{pname}' declares no domain, so it is not attached "1100                    "to any node shape. Declare a domain, or pass "1101                    "attach_domainless_properties=True to restore the old behaviour."1102                )

1104    def _build_property_shape(self, prop: Dict[str, Any]) -> PropertyShape:1105        ptype = prop.get("type", "")1106        range_ = prop.get("range", "")1107        if isinstance(range_, list):1108            range_ = range_[0] if range_ else ""11091110        cardinality = prop.get("cardinality") or {}1111        min_count = cardinality.get("min") if isinstance(cardinality, dict) else None1112        max_count = cardinality.get("max") if isinstance(cardinality, dict) else None11131114        if prop.get("required") and min_count is None:1115            min_count = 111161117        datatype = None1118        class_ = None1119        if ptype in ("datatype", "data", "DatatypeProperty"):1120            datatype = self._resolve_xsd(range_) if range_ else None1121        elif ptype in ("object", "ObjectProperty"):1122            class_ = range_ if range_ else None11231124        in_values = (1125            prop.get("one_of") or prop.get("enum") or prop.get("allowed_values")1126        )1127        if in_values and self.quality_tier in ("standard", "strict"):1128            in_values = list(in_values)1129        else:1130            in_values = None11311132        pattern = prop.get("pattern") if self.quality_tier in ("standard", "strict") else None11331134        return PropertyShape(1135            path=prop.get("name", ""),1136            name=prop.get("label") or prop.get("name"),1137            description=prop.get("description") or prop.get("comment"),1138            datatype=datatype,1139            class_=class_,1140            min_count=min_count,1141            max_count=max_count,1142            in_values=in_values,1143            has_value=prop.get("has_value"),1144            pattern=pattern,1145            severity=self.severity,1146        )

1148    def _propagate_inheritance(1149        self, graph: SHACLGraph, class_index: Dict[str, Dict[str, Any]]1150    ) -> None:1151        shape_by_class = {ns.target_class: ns for ns in graph.node_shapes}11521153        for _ in range(20):  # max 20 passes; stops early when stable1154            changed = False1155            for node_shape in graph.node_shapes:1156                cls_data = class_index.get(node_shape.target_class, {})1157                parent_name = cls_data.get("parent") or cls_data.get("parent_class")1158                if not parent_name or parent_name not in shape_by_class:1159                    continue1160                parent_shape = shape_by_class[parent_name]1161                existing_paths = {ps.path for ps in node_shape.property_shapes}1162                for pps in parent_shape.property_shapes:1163                    if pps.path not in existing_paths:1164                        node_shape.property_shapes.append(dataclass_replace(pps))1165                        existing_paths.add(pps.path)1166                        changed = True1167            if not changed:1168                break

1170    def _apply_quality_tier(self, graph: SHACLGraph) -> None:1171        if self.quality_tier == "strict":1172            for node_shape in graph.node_shapes:1173                # Only close shapes that declare at least one property1174                if node_shape.property_shapes:1175                    node_shape.closed = True

1182    def _term_iri(self, graph: SHACLGraph, local: str, kind: str = "class") -> str:1183        """1184        Resolve a class or property name to the absolute IRI the data uses.11851186        Every serializer goes through here. Turtle alone was corrected at first,1187        which left JSON-LD and N-Triples still pasting names onto the shapes1188        namespace, so the shapes they produced went on matching nothing (#1104).11891190        `kind` selects the index: a property may share a class's name, and the1191        two can carry different IRIs.1192        """1193        if local.startswith("http://") or local.startswith("https://"):1194            return local1195        index = graph.property_iris if kind == "property" else graph.class_iris1196        resolved = index.get(local)1197        if resolved:1198            return resolved1199        # Fall back to the other index before giving up: sh:class names a class,1200        # but a caller may pass a term only registered on the other side.1201        other = graph.class_iris if kind == "property" else graph.property_iris1202        resolved = other.get(local)1203        if resolved:1204            return resolved1205        if ":" in local:1206            return local1207        separator = "" if graph.base_uri.endswith(("#", "/", ":")) else "#"1208        return f"{graph.base_uri}{separator}{local}"

1234            if node_shape.closed:1235                block.append("    sh:closed true ;")1236                block.append("    sh:ignoredProperties ( <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> ) ;")12371238            for i, ps in enumerate(node_shape.property_shapes):1239                is_last = i == len(node_shape.property_shapes) - 11240                terminator = " ." if is_last else " ;"

340        # Placeholder implementation for now341        # In a real implementation, this would load owlready2 or similar⋯    # ... 空行略 ...·        try:·            if isinstance(ontology, dict):·                self._validate_structure(ontology, result)⋯            # ... 空行略 ...·            # Simulate reasoning checks·            if self.check_consistency:·                # Logic to check consistency would go here·                pass⋯            # ... 空行略 ...352            if self.check_satisfiability:353                # Logic to check satisfiability would go here354                pass

290@dataclass291class ValidationResult:292    """Result of an ontology validation operation."""293    valid: bool = True294    consistent: bool = True295    satisfiable: bool = True296    errors: List[str] = field(default_factory=list)297    warnings: List[str] = field(default_factory=list)

167    try:168        import pyshacl169    except ImportError as exc:170        raise ImportError(171            "pyshacl is required for SHACL validation. "172            "Install it with: pip install semantica[shacl]"173        ) from exc174175    try:176        import rdflib177    except ImportError as exc:178        raise ImportError(179            "rdflib is required for SHACL validation. "180            "Install it with: pip install rdflib"181        ) from exc182183    data_g = rdflib.Graph()184    data_g.parse(data=data_graph_str, format=data_graph_format)185186    _fmt_map = {187        "turtle": "turtle", "ttl": "turtle",188        "json-ld": "json-ld", "jsonld": "json-ld", "json_ld": "json-ld",189        "n-triples": "nt", "ntriples": "nt", "nt": "nt",190    }191    shacl_g = rdflib.Graph()192    shacl_g.parse(data=shacl_str, format=_fmt_map.get(shacl_format.lower().strip(), shacl_format))193194    conforms, results_graph, results_text = pyshacl.validate(195        data_g,196        shacl_graph=shacl_g,197        inference="none",198        abort_on_first=False,199    )

82    def explain_violations(self) -> None:83        """Populate a plain-English explanation on every violation. No LLM call."""84        _TEMPLATES = {85            "MinCountConstraintComponent": (86                "Node <{focus_node}> is missing required property <{path}>. "87                "At least {min_count} value(s) are required."88            ),89            "MaxCountConstraintComponent": (90                "Node <{focus_node}> has too many values for <{path}>. "91                "At most {max_count} value(s) are allowed."92            ),93            "DatatypeConstraintComponent": (94                "Node <{focus_node}> has value '{value}' for <{path}> "95                "but the expected datatype is {datatype}."96            ),97            "ClassConstraintComponent": (98                "Node <{focus_node}> has value '{value}' for <{path}> "99                "but it must be an instance of {class_}."100            ),101            "InConstraintComponent": (102                "Node <{focus_node}> has value '{value}' for <{path}> "103                "which is not in the allowed set."104            ),105            "PatternConstraintComponent": (106                "Node <{focus_node}> has value '{value}' for <{path}> "107                "which does not match the required pattern."108            ),109            "ClosedConstraintComponent": (110                "Node <{focus_node}> has undeclared property <{path}> "111                "which is not allowed by the closed shape."112            ),113        }

223    def build_concept_scheme_uri(self, name: str) -> str:224        """225        Build a ConceptScheme URI anchored at the current base URI.226227        The scheme name is slugified (spaces → hyphens, lower-cased) so that228        ``"My Vocabulary"`` becomes ``<base>/vocab/my-vocabulary>``.229230        Args:231            name: Human-readable vocabulary name232233        Returns:234            ConceptScheme URI string235        """236        slug = re.sub(r"[^a-zA-Z0-9]+", "-", name).strip("-").lower()237        return urljoin(self.get_base_uri(), f"vocab/{slug}")

249        return {250            #OWL alignments251            "equivalentClass": f"{owl_ns}equivalentClass",252            "equivalentProperty": f"{owl_ns}equivalentProperty",253            "sameAs": f"{owl_ns}sameAs",254            #SKOS alignments255            "exactMatch": f"{skos_ns}exactMatch",256            "closeMatch": f"{skos_ns}closeMatch",257            "broadMatch": f"{skos_ns}broadMatch",258            "narrowMatch": f"{skos_ns}narrowMatch",259            "relatedMatch": f"{skos_ns}relatedMatch",

6GRAPH_EXTRACTION_PROMPT = """7-Goal-8Given a text document that is potentially relevant to this activity and a list of entity types, identify all entities of those types from the text and all relationships among the identified entities.⋯# ... 空行略 ...10-Steps-111. Identify all entities. For each identified entity, extract the following information:12- entity_name: Name of the entity, capitalized13- entity_type: One of the following types: [{entity_types}]14- entity_description: Comprehensive description of the entity's attributes and activities15Format each entity as ("entity"<|><entity_name><|><entity_type><|><entity_description>)

60        self, text: str, entity_types: list[str], source_id: str61    ) -> tuple[pd.DataFrame, pd.DataFrame]:62        """Extract entities and relationships from the supplied text."""63        try:64            # Invoke the entity extraction65            result = await self._process_document(text, entity_types)66        except Exception as e:  # pragma: no cover - defensive logging67            logger.exception("error extracting graph")68            self._on_error(69                e,70                traceback.format_exc(),71                {72                    "source_id": source_id,73                    "text": text,74                },75            )76            return _empty_entities_df(), _empty_relationships_df()7778        return self._process_result(79            result,80            source_id,81            TUPLE_DELIMITER,82            RECORD_DELIMITER,83        )8485    async def _process_document(self, text: str, entity_types: list[str]) -> str:86        messages_builder = CompletionMessagesBuilder().add_user_message(87            self._extraction_prompt.format(**{88                INPUT_TEXT_KEY: text,89                ENTITY_TYPES_KEY: ",".join(entity_types),90            })91        )

124    def _process_result(125        self,126        result: str,127        source_id: str,128        tuple_delimiter: str,129        record_delimiter: str,130    ) -> tuple[pd.DataFrame, pd.DataFrame]:131        """Parse the result string into entity and relationship data frames."""132        entities: list[dict[str, Any]] = []133        relationships: list[dict[str, Any]] = []134135        records = [r.strip() for r in result.split(record_delimiter)]136137        for raw_record in records:138            record = re.sub(r"^\(|\)$", "", raw_record.strip())139            if not record or record == COMPLETION_DELIMITER:140                continue141142            record_attributes = record.split(tuple_delimiter)143            record_type = record_attributes[0]144145            if record_type == '"entity"' and len(record_attributes) >= 4:146                entity_name = clean_str(record_attributes[1].upper())147                entity_type = clean_str(record_attributes[2].upper())148                entity_description = clean_str(record_attributes[3])149                entities.append({150                    "title": entity_name,151                    "type": entity_type,152                    "description": entity_description,

12@dataclass13class Community(Named):14    """A protocol for a community in the system."""1516    level: str17    """Community level."""1819    parent: str20    """Community ID of the parent node of this community."""2122    children: list[str]23    """List of community IDs of the child nodes of this community."""2425    entity_ids: list[str] | None = None26    """List of entity IDs related to the community (optional)."""2728    relationship_ids: list[str] | None = None29    """List of relationship IDs related to the community (optional)."""

12@dataclass13class CommunityReport(Named):14    """Defines an LLM-generated summary report of a community."""1516    community_id: str17    """The ID of the community this report is associated with."""1819    summary: str = ""20    """Summary of the report."""2122    full_content: str = ""23    """Full content of the report."""2425    rank: float | None = 1.026    """Rank of the report, used for sorting (optional). Higher means more important"""2728    full_content_embedding: list[float] | None = None29    """The semantic (i.e. text) embedding of the full report content (optional)."""3031    attributes: dict[str, Any] | None = None32    """A dictionary of additional attributes associated with the report (optional)."""3334    size: int | None = None35    """The size of the report (Amount of text units)."""3637    period: str | None = None38    """The period of the report (optional)."""

74    async def __call__(self, input_text: str):75        """Call method definition."""76        output = None77        try:78            prompt = self._extraction_prompt.format(**{79                INPUT_TEXT_KEY: input_text,80                MAX_LENGTH_KEY: str(self._max_report_length),81            })82            response = await self._model.completion_async(83                messages=prompt,84                response_format=CommunityReportResponse,  # A model is required when using json mode85            )8687            output = response.formatted_response  # type: ignore88        except Exception as e:89            logger.exception("error generating community report")90            self._on_error(e, traceback.format_exc(), None)9192        text_output = self._get_text_output(output) if output else ""93        return CommunityReportsResult(94            structured_output=output,95            output=text_output,96        )