o á,Ëh"oã@sndZddlZddlZddlZddlmZddlZddlZ ddl Z ddl Z ddl Z ddl mZGdd„dƒZdS)z¡ Created by Jaided AI Released Date: 18/08/2022 Description: DBNet text detection module. Many parts of the codes are adapted from https://github.com/MhLiao/DB éN)ÚPolygoné)Ú Configurablec@sðeZdZ       d3dd„Zd d „Zd d „Zd d„Zdd„Zdd„Zd4dd„Z dd„Z dd„Z d5dd„Z d4dd„Z     d6d d!„Zd"d#„Z   d7d$d%„Z   d7d&d'„Zd8d)d*„Zd+d,„Zd-d.„Zd/d0„Z       d9d1d2„ZdS):ÚDBNetÚresnet18NÚ pretrainedTÚcudarc Cs~||_tj tj t¡dd¡}t|dƒ} t | ¡|_ Wdƒn1s%wY|dur6|  |j |¡|_ ||j   ¡vrA||_ n t d d |j   ¡¡¡ƒ‚|durV||_n tj tj t¡d¡|_|r¥||j |d  ¡vrtj |j|j |d|¡} d } n tj |j|¡} d } tj | ¡s™t|  || ¡ƒ‚| |j |d | ¡nd|_t |j d ¡|_|j d |_|j d|_dS)a DBNet text detector class Parameters ---------- backbone : str, optional Backbone to use. Options are "resnet18" and "resnet50". The default is "resnet18". weight_dir : str, optional Path to directory that contains weight files. If set to None, the path will be set to "../weights/". The default is None. weight_name : str, optional Name of the weight to use as specified in DBNet_inference.yaml or a filename in weight_dir. The default is 'pretrained'. initialize_model : Boolean, optional If True, construct the model and load weight at class initialization. Otherwise, only initial the class without constructing the model. The default is True. dynamic_import_relative_path : str, optional Relative path to 'model/detector.py'. This option is for supporting integrating this module into other modules. For example, easyocr/DBNet This should be left as None when calling this module as a standalone. The default is None. device : str, optional Device to use. Options are "cuda" and "cpu". The default is 'cuda'. verbose : int, optional Verbosity level. The default is 0. Raises ------ ValueError Raised when backbone is invalid. FileNotFoundError Raised when weight file is not found. Returns ------- None. ÚconfigszDBNet_inference.yamlÚrNz2Invalid backbone. Current support backbone are {}.ú,ÚweightsÚweightzUA weight with a name {} is found in DBNet_inference.yaml but cannot be find file: {}.zYA weight with a name {} is not found in DBNet_inference.yaml and cannot be find file: {}.ÚmodelÚBGR_MEANÚmin_detection_sizeÚmax_detection_size)ÚdeviceÚosÚpathÚjoinÚdirnameÚ__file__ÚopenÚyamlÚ safe_loadr Úset_relative_import_pathÚkeysÚbackboneÚ ValueErrorÚformatÚ weight_dirÚisfileÚFileNotFoundErrorÚinitialize_modelrÚnpÚarrayrrr) Úselfrr Ú weight_namer#Údynamic_import_relative_pathrÚverboseÚ config_pathÚfidÚ weight_pathÚ error_message©r.úN/var/www/html/scripts/venv/lib/python3.10/site-packages/easyocr/DBNet/DBNet.pyÚ__init__s4. ÿ  zDBNet.__init__c Csp|dusJ‚| tj¡}| ¡D]%\}}|dkr(| |d || d¡¡i¡qt|tƒr4| ||¡}q q|S)a¯ Create relative import paths for modules specified in class. This method is recursive. Parameters ---------- configs : dict Configuration dictionary from .yaml file. dynamic_import_relative_path : str, optional Relative path to 'model/detector/'. This option is for supporting integrating this module into other modules. For example, easyocr/DBNet This should be left as None when calling this module as a standalone. The default is None. Returns ------- configs : dict Configuration dictionary with correct relative path. NÚclassÚ.) ÚsplitrÚsepÚitemsÚupdaterÚ isinstanceÚdictr)r&r r(ÚpreficesÚkeyÚvaluer.r.r/rjs    zDBNet.set_relative_import_pathcCs<|jdur tdƒ‚|jjtj||jddd|j ¡dS)a. Load weight to model. Parameters ---------- weight_path : str Path to trained weight. Raises ------ RuntimeError Raised when the model has not yet been contructed. Returns ------- None. Nz#model has not yet been constructed.)Ú map_locationF)Ústrict)rÚ RuntimeErrorÚload_state_dictÚtorchÚloadrÚeval)r&r,r.r.r/Ú load_weightŠs zDBNet.load_weightcCst |¡jj |j¡|_dS)zë Contruct text detection model based on the configuration in .yaml file. Parameters ---------- config : dict Configuration dictionary. Returns ------- None. N)rÚconstruct_class_from_configÚ structureÚbuilderÚbuildrr)r&Úconfigr.r.r/Úconstruct_model¡s zDBNet.construct_modelcCsR| |¡| |¡t|jjtjjƒr%|jdkr'|jjj  |j¡|j_dSdSdS)aO Wrapper to initialize text detection model. This model includes contructing and weight loading. Parameters ---------- model_config : dict Configuration dictionary. weight_path : str Path to trained weight. Returns ------- None. ÚcpuN) rIrCr7rr@ÚnnÚ DataParallelrÚmoduleÚto)r&Ú model_configr,r.r.r/r#°s  ÿzDBNet.initialize_modelcCs’t|tƒrtj |¡rt |tj¡ d¡}|St d  |¡ƒ‚t|t j ƒr+| d¡}|St|t jjƒrEt  |¡dd…dd…ddd…f}|Stdƒ‚)a Load or convert input to OpenCV BGR image numpy array. Parameters ---------- image : str, PIL.Image, or np.ndarray Image to load or convert. Raises ------ FileNotFoundError Raised when the input is a path to file (str), but the file is not found. TypeError Raised when the data type of the input is not supported. Returns ------- image : np.ndarray OpenCV BGR image. Úfloat32zCannot find {}NéÿÿÿÿzYUnsupport image format. Only path-to-file, opencv BGR image, and PIL image are supported.)r7Ústrrrr!Úcv2ÚimreadÚ IMREAD_COLORÚastyper"rr$ÚndarrayÚPILÚImageÚasarrayÚ TypeError©r&Úimager.r.r/Ú get_cv2_imageÅs   ø  û"þzDBNet.get_cv2_imagec Cs¸|j\}}}|durt|jt|||jƒƒ}||kr4tt |d¡dƒ}tt |||d¡dƒ}ntt |d¡dƒ}tt |||d¡dƒ}t  |||f¡}|||ffS)a Resize image such that the shorter side of the image is equal to the closest multiple of 32 to the provided detection_size. If detection_size is not provided, it will be resized to the closest multiple of 32 each side. If the original size exceeds the min-/max-detection sizes (specified in configs.yaml), it will be resized to be within the min-/max-sizes. Parameters ---------- img : np.ndarray OpenCV BGR image. detection_size : int, optional Target detection size. The default is None. Returns ------- np.ndarray Resized OpenCV BGR image. The width and height of this image should be multiple of 32. Né ) ÚshapeÚmaxrÚminrÚintÚmathÚceilrSÚresize) r&ÚimgÚdetection_sizeÚheightÚwidthÚ_Ú new_heightÚ new_widthÚ resized_imgr.r.r/Ú resize_imageès   zDBNet.resize_imagecCst |¡ ddd¡ ¡ d¡S)a4 Convert image array (assuming OpenCV BGR format) to image tensor. Parameters ---------- image : np.ndarray OpenCV BGR image. Returns ------- torch.tensor Tensor image with 4 dimension [batch, channel, width, height]. érr)r@Ú from_numpyÚpermuteÚfloatÚ unsqueezer\r.r.r/Úimage_array2tensor szDBNet.image_array2tensorcCs||jdS)zÿ Normalize image by substracting BGR mean and divided by 255 Parameters ---------- image : np.ndarray OpenCV BGR image. Returns ------- np.ndarray OpenCV BGR image. gào@)rr\r.r.r/Únormalize_imageszDBNet.normalize_imagecCs8| |¡}|j||d\}}| |¡}| |¡}||fS)a$ Wrapper to load and convert an image to an image tensor Parameters ---------- image : path-to-file, PIL.Image, or np.ndarray Image to load or convert. detection_size : int, optional Target detection size. The default is None. Returns ------- img : torch.tensor Tensor image with 4 dimension [batch, channel, width, height].. original_shape : tuple A tuple (height, width) of the original input image before resizing. ©rh)r^rorvru)r&Ú image_pathrhrgÚoriginal_shaper.r.r/Ú load_image,s   zDBNet.load_imagecs.t‡‡fdd„|DƒŽ\}}tj|dd|fS)a¨ Wrapper to load or convert list of multiple images to a single image tensor. Multiple images are concatenated together on the first dimension. Parameters ---------- images : a list of path-to-file, PIL.Image, or np.ndarray Image to load or convert. detection_size : int, optional Target detection size. The default is None. Returns ------- img : torch.tensor A single tensor image with 4 dimension [batch, channel, width, height]. original_shape : tuple A list of tuples (height, width) of the original input image before resizing. csg|] }ˆj|ˆd‘qS)rw)rz)Ú.0r]©rhr&r.r/Ú Xsÿz%DBNet.load_images..r)Údim)Úzipr@Úcat)r&ÚimagesrhÚoriginal_shapesr.r|r/Ú load_imagesEs ÿzDBNet.load_imagesçš™™™™™É?éFc  Cs¼|j||d} g} g} t| d¡ƒD]9} || \} }|r/|j|| | | || |||d\}}n|j|| | | || |||d\}}|  |¡|  |¡qtdd„t| | ƒDƒŽ\} } | | fS)a Translate probability heatmap tensor to text region boudning boxes. Parameters ---------- image_tensor : torch.tensor Image tensor. original_shapes : tuple Original size of the image (height, width) of the input image (before rounded to the closest multiple of 32). hmap : torch.tensor Probability heatmap tensor. text_threshold : float, optional Minimum probability for each pixel of heatmap tensor to be considered as a valid text pixel. The default is 0.2. bbox_min_score : float, optional Minimum score for each detected bounding box to be considered as a valid text bounding box. The default is 0.2. bbox_min_size : int, optional Minimum size for each detected bounding box to be considered as a valid text bounding box. The default is 3. max_candidates : int, optional Maximum number of detected bounding boxes to be considered as candidates for valid text bounding box. Setting it to 0 implies no maximum. The default is 0. as_polygon : boolean, optional If True, return the bounding box as polygon (fine vertrices), otherwise return as rectangular. The default is False. Returns ------- boxes_batch : list of lists Bounding boxes of each text box. scores_batch : list of floats Confidence scores of each text box. )Ú thresholdr)Úbbox_min_scoreÚ bbox_min_sizeÚmax_candidatescSs<g|]\}}t|dkƒrtdd„t||ƒDƒŽnddg‘qS)rcSs g|] \}}|dkr||f‘qS©rr.)r{ÚboxÚscorer.r.r/r}¥sÿz.DBNet.hmap2bbox...r.)Úanyr)r{ÚboxesÚscoresr.r.r/r}¥s ÿ þÿþz#DBNet.hmap2bbox..)ÚbinarizeÚrangeÚsizeÚpolygons_from_bitmapÚboxes_from_bitmapÚappendr)r&Ú image_tensorr‚ÚhmapÚtext_thresholdr‡rˆr‰Ú as_polygonÚ segmentationÚ boxes_batchÚ scores_batchÚ batch_indexrirjrŽrr.r.r/Ú hmap2bbox\s<.  ù  ù  ýzDBNet.hmap2bboxcCs||kS)a Apply threshold to return boolean tensor. Parameters ---------- tensor : torch.tensor input tensor. threshold : float Threshold. Returns ------- torch.tensor Boolean tensor. r.)r&Útensorr†r.r.r/r­szDBNet.binarizecCsÜ| d¡dks J‚| ¡ ¡d}| ¡ ¡ ¡d}|j\} } g} g} t |d tj ¡tj tj ¡\} }|dkr?| d|…} | D]¨}dt  |d¡}t  ||d¡}| d¡}|jddkr_qA| || d d ¡¡}||krnqA|jdd kr„|j|d d }t|ƒdkrƒqAnqA| d d ¡}| | d ¡¡\}}||d krœqAt|tƒs©| ¡}| ¡}t t |dd…df| |¡d|¡|dd…df<t t |dd…df| |¡d|¡|dd…df<|  | ¡¡|  |¡qA| | fS)a¬ Translate boolean tensor to fine polygon indicating text bounding boxes Parameters ---------- hmap : torch.tensor Probability heatmap tensor. segmentation : torch.tensor Segmentataion tensor. dest_width : TYPE target width of the output. dest_height : TYPE target width of the output. bbox_min_score : float, optional Minimum score for each detected bounding box to be considered as a valid text bounding box. The default is 0.2. bbox_min_size : int, optional Minimum size for each detected bounding box to be considered as a valid text bounding box. The default is 3. max_candidates : int, optional Maximum number of detected bounding boxes to be considered as candidates for valid text bounding box. Setting it to 0 implies no maximum. The default is 0. Returns ------- boxes_batch : list of lists Polygon bounding boxes of each text box. scores_batch : list of floats Confidence scores of each text box. rréÿNgü©ñÒMb`?T)rQrpérQrpg@)Ú unclip_ratio)rQrrp)r’rJÚnumpyÚdetachr`rSÚ findContoursrVr$Úuint8Ú RETR_LISTÚCHAIN_APPROX_SIMPLEÚ arcLengthÚ approxPolyDPÚreshapeÚbox_score_fastÚunclipÚlenÚget_mini_boxesr7rcÚitemÚclipÚroundr•Útolist)r&r—ršÚ dest_widthÚ dest_heightr‡rˆr‰ÚbitmaprirjrŽrÚcontoursrkÚcontourÚepsilonÚapproxÚpointsrŒr‹Ússider.r.r/r“ÀsV( þ   ÿ    ÿ ÿ zDBNet.polygons_from_bitmapcCsî| d¡dks J‚| ¡ ¡d}| ¡ ¡ ¡d}|j\} } t |d tj ¡tj tj ¡\} } |dkr=t t | ƒ|ƒ} nt | ƒ} tj| ddftjd}tj| ftjd}t| ƒD]—}| |}| |¡\}}||krkqYt |¡}| || dd¡¡}||krqY| |¡ ddd¡}| |¡\}}||dkr—qYt |¡}t|tƒs©| ¡}| ¡}t t |dd…df| |¡d|¡|dd…df<t t |dd…df| |¡d|¡|dd…df<| tj¡||dd…dd…f<|||<qY| ¡|fS) a« Translate boolean tensor to fine polygon indicating text bounding boxes Parameters ---------- hmap : torch.tensor Probability heatmap tensor. segmentation : torch.tensor Segmentataion tensor. dest_width : TYPE target width of the output. dest_height : TYPE target width of the output. bbox_min_score : float, optional Minimum score for each detected bounding box to be considered as a valid text bounding box. The default is 0.2. bbox_min_size : int, optional Minimum size for each detected bounding box to be considered as a valid text bounding box. The default is 3. max_candidates : int, optional Maximum number of detected bounding boxes to be considered as candidates for valid text bounding box. Setting it to 0 implies no maximum. The default is 0. Returns ------- boxes_batch : list of lists Polygon bounding boxes of each text box. scores_batch : list of floats Confidence scores of each text box. rrr r¡rp©ÚdtyperQN)r’rJr£r¤r`rSr¥rVr$r¦r§r¨rbr®ÚzerosÚint16rPr‘r¯r%r¬r«r­r7rcr°r±r²r³)r&r—ršr´rµr‡rˆr‰r¶rirjr·rkÚ num_contoursrŽrÚindexr¸r»r¼rŒr‹r.r.r/r”sN' þ      ÿ ÿ  zDBNet.boxes_from_bitmapçø?cCsFt|ƒ}|j||j}t ¡}| |tjtj¡t  |  |¡¡}|S©N) rÚareaÚlengthÚ pyclipperÚPyclipperOffsetÚAddPathÚJT_ROUNDÚET_CLOSEDPOLYGONr$r%ÚExecute)r&r‹r¢ÚpolyÚdistanceÚoffsetÚexpandedr.r.r/r­os z DBNet.unclipc Cs°t |¡}ttt |¡ƒdd„d}d\}}}}|dd|ddkr)d}d}nd}d}|dd|ddkr>d}d}nd}d}||||||||g}|t|dƒfS) NcSs|dS)Nrr.)Úxr.r.r/Úzsz&DBNet.get_mini_boxes..)r:)rrrpr…rrr…rp)rSÚ minAreaRectÚsortedÚlistÚ boxPointsrb) r&r¸Ú bounding_boxr»Úindex_1Úindex_2Úindex_3Úindex_4r‹r.r.r/r¯xs"    ÿzDBNet.get_mini_boxesc Cs†|jdd…\}}| ¡}t t |dd…df ¡¡ tj¡d|d¡}t t |dd…df  ¡¡ tj¡d|d¡}t t |dd…df ¡¡ tj¡d|d¡}t t |dd…df  ¡¡ tj¡d|d¡} tj | |d||dftj d} |dd…df||dd…df<|dd…df||dd…df<t   | | ddd¡ tj¡d¡t  ||| d…||d…f| ¡dS)a- Calculate total score of each bounding box Parameters ---------- hmap : torch.tensor Probability heatmap tensor. box_ : list Rectanguar bounding box. Returns ------- float Confidence score. Nrprrr½rQ)r`Úcopyr$r±ÚfloorrbrVÚint32rerar¿r¦rSÚfillPolyr«Úmean) r&r—Úbox_ÚhÚwr‹ÚxminÚxmaxÚyminÚymaxÚmaskr.r.r/r¬s0000$   (zDBNet.box_score_fastcCs|jj|ddS)ap Run the model to obtain a heatmap tensor from a image tensor. The heatmap tensor indicates the probability of each pixel being a part of text area. Parameters ---------- image_tensor : torch.tensor Image tensor. Returns ------- torch.tensor Probability heatmap tensor. F)Útraining)rÚforward)r&r–r.r.r/Ú image2hmap­szDBNet.image2hmapc  Cs‚t|tƒs|g}|j||d\} } t ¡| | ¡} |j| | | |||||d\} } Wdƒn1s4wY|r?| | fS| S)az Wrapper to run the model on an input image to get text bounding boxes. Parameters ---------- image : path-to-file, PIL.Image, or np.ndarray Image to load or convert. text_threshold : float, optional Minimum probability for each pixel of heatmap tensor to be considered as a valid text pixel. The default is 0.2. bbox_min_score : float, optional Minimum score for each detected bounding box to be considered as a valid text bounding box. The default is 0.2. bbox_min_size : int, optional Minimum size for each detected bounding box to be considered as a valid text bounding box. The default is 3. max_candidates : int, optional Maximum number of detected bounding boxes to be considered as candidates for valid text bounding box. Setting it to 0 implies no maximum. The default is 0. detection_size : int, optional Target detection size. Please see docstring under method resize_image() for explanation. The default is None. as_polygon : boolean, optional If true, return the bounding boxes as find polygons, otherwise, return as rectagular. The default is False. return_scores : boolean, optional If true, return confidence score along with the text bounding boxes. The default is False. Returns ------- list of lists Text bounding boxes. If return_scores is set to true, another list of lists will also be returned. rw)r˜r‡rˆr‰r™N)r7rÕrƒr@Úno_gradrërž)r&r]r˜r‡rˆr‰rhr™Ú return_scoresr–r‚r—Ú batch_boxesÚ batch_scoresr.r.r/Ú inference¾s$ .   ùþ zDBNet.inference)rNrTNrrrÄrŠ)r„r„r…rF)r„r…r)rÃ)r„r„r…rNFF)Ú__name__Ú __module__Ú __qualname__r0rrCrIr#r^rorurvrzrƒržrr“r”r­r¯r¬rërðr.r.r.r/rs\ ùU  #$   øQ ù` ù T ør)Ú__doc__rrdrÚshapely.geometryrÚ PIL.ImagerXr£r$rSrÇr@Úmodel.constructorrrr.r.r.r/Ús