pycocoevalcap代码的使用和问题解决网站首页 技术教程

pycocoevalcap代码的使用和问题解决

渡口五十六 2023-06-22 04:00:02

简介pycocoevalcap代码的使用和问题解决

github链接

https://github.com/232525/PureT

总结

按照下面改了好多，总是出错，最后我重启了电脑，并使用通过pip安装的pycocoevalcap和java1.8能够完整运行，nice，真是无语

pycocoevalcap             1.2                      pypi_0    pypi
pycocotools               2.0.6                    pypi_0    pypi
pycocotools-windows       2.0.0.2                  pypi_0    pypi

java的下载 https://blog.csdn.net/JunLeon/article/details/122623465

下载官网：Java Downloads | Oracle
下载版本：jdk-8u321-windows-x64.exe
百度网盘：链接：https://pan.baidu.com/s/1F43G56vjd5JtZXxbkQJlIA   提取码：8888

windows安装wegt
windows10下运行.sh文件报错：wget: command not found
https://blog.csdn.net/buluxianfeng/article/details/116049447

按照readme.md处理

Demo

https://blog.csdn.net/weixin_41848012/article/details/121254472
https://zhuanlan.zhihu.com/p/613433647

# -*- coding=utf-8 -*-
# author: w61
# Test for several ways to compute the score of the generated words.
from pycocoevalcap.bleu.bleu import Bleu
from pycocoevalcap.meteor.meteor import Meteor
from pycocoevalcap.rouge.rouge import Rouge
from pycocoevalcap.cider.cider import Cider
from pycocoevalcap.spice.spice import Spice
from pycocoevalcap.wmd.wmd import WMD

class Scorer():
    def __init__(self,ref,gt):
        self.ref = ref
        self.gt = gt
        print('setting up scorers...')
        self.scorers = [
            (Bleu(4), ["Bleu_1", "Bleu_2", "Bleu_3", "Bleu_4"]),
            (Meteor(),"METEOR"),
            (Rouge(), "ROUGE_L"),
            (Cider(), "CIDEr"),
            (Spice(), "SPICE"),
            (WMD(),   "WMD"),
        ]
    
    def compute_scores(self):
        total_scores = {}
        for scorer, method in self.scorers:
            print('computing %s score...'%(scorer.method()))
            score, scores = scorer.compute_score(self.gt, self.ref)
            if type(method) == list:
                for sc, scs, m in zip(score, scores, method):
                    print("%s: %0.3f"%(m, sc))
                total_scores["Bleu"] = score
            else:
                print("%s: %0.3f"%(method, score))
                total_scores[method] = score
        
        print('*****DONE*****')
        for key,value in total_scores.items():
            print('{}:{}'.format(key,value))

if __name__ == '__main__':
    ref = {
        '1':['go down the stairs and stop at the bottom .'],
        '2':['this is a cat.']
    }
    gt = {
        '1':['Walk down the steps and stop at the bottom. ', 'Go down the stairs and wait at the bottom.','Once at the top of the stairway, walk down the spiral staircase all the way to the bottom floor. Once you have left the stairs you are in a foyer and that indicates you are at your destination.'],
        '2':['It is a cat.','There is a cat over there.','cat over there.']
    }
    # 注意，这里如果只有一个sample，cider算出来会是0，详情请看评论区。
    scorer = Scorer(ref,gt)
    scorer.compute_scores()

自己通过pip install的可能没有WMD,可以将其注释掉

问题

cocoEvalCapDemo.ipynb

1、发生异常: FileNotFoundError (note: full exception trace is shown but execution is paused at: _run_module_as_main)

[WinError 2] 系统找不到指定的文件。

执行my_image_captioning_CNNimage_captioning_to_cnncoco_captioncocoEvalCapDemo.ipynb的时候出现的错误。

错误代码如下；

# create cocoEval object by taking coco and cocoRes
cocoEval = COCOEvalCap(coco, cocoRes)

# evaluate on a subset of images by setting
# cocoEval.params['image_id'] = cocoRes.getImgIds()
# please remove this line when evaluating the full validation set
cocoEval.params['image_id'] = cocoRes.getImgIds()

# evaluate results
# SPICE will take a few minutes the first time, but speeds up due to caching
cocoEval.evaluate()  -------------------------------------------------------------------》错误
|
|
|
    def evaluate(self):
        imgIds = self.params['image_id']
        # imgIds = self.coco.getImgIds()
        gts = {}
        res = {}
        for imgId in imgIds:
            gts[imgId] = self.coco.imgToAnns[imgId]
            res[imgId] = self.cocoRes.imgToAnns[imgId]

        # =================================================
        # Set up scorers
        # =================================================
        print('tokenization...')
        tokenizer = PTBTokenizer()
        gts  = tokenizer.tokenize(gts)  ------------------------------------------------》错误
        res = tokenizer.tokenize(res)
|
|
|

cmd.append(os.path.basename(tmp_file.name))
p_tokenizer = subprocess.Popen(cmd, cwd=path_to_jar_dirname, stdout=subprocess.PIPE)
		------------------------------------------------------------------------------》 错误

错误信息如下：
发生异常: FileNotFoundError       (note: full exception trace is shown but execution is paused at: _run_module_as_main)
[WinError 2] 系统找不到指定的文件。

参考解决的方法的网址：https://stackoverflow.com/questions/73193119/python-filenotfounderror-winerror-2-the-system-cannot-find-the-file-specifie

解决方法：
1、安装Java JDK1.8
2、将subprocess.py中找到如下代码

def __init__(self, args, bufsize=-1, executable=None,
             stdin=None, stdout=None, stderr=None,
             preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS,
             shell=False, cwd=None, env=None, universal_newlines=False,
             startupinfo=None, creationflags=0,
             restore_signals=True, start_new_session=False,
             pass_fds=(), *, encoding=None, errors=None):

将shell=False更改为：shell=True

完美解决

2、发生异常: OSError (note: full exception trace is shown but execution is paused at: _run_module_as_main)

[Errno 22] Invalid argument

计算computing METEOR score…的时候爆出的错误‘
https://github.com/wangleihitcs/CaptionMetrics/issues/1

对于以上等问题的解决

安装java后重启电脑
https://zhuanlan.zhihu.com/p/159204903

https://blog.csdn.net/Bit_Coders/article/details/120840271

METEOR score
https://github.com/wangleihitcs/CaptionMetrics/issues/1

https://github.com/tylin/coco-caption/issues/8

修改后的pycocoevalcap(我个人没有验证)
https://github.com/salaniz/pycocoevalcap

Spice

Threads( StanfordCoreNLP ) [0.667 seconds]
Error: Could not score batched file input:
org.fusesource.lmdbjni.LMDBException: 磁盘空间不足。

        at org.fusesource.lmdbjni.Util.checkErrorCode(Util.java:44)
        at org.fusesource.lmdbjni.Env.open(Env.java:192)
        at org.fusesource.lmdbjni.Env.open(Env.java:72)
        at org.fusesource.lmdbjni.Env.open(Env.java:65)
        at edu.anu.spice.LmdbTupleDB.putTransaction(LmdbTupleDB.java:69)
        at edu.anu.spice.SpiceParser.loadTuplesFromDB(SpiceParser.java:216)
        at edu.anu.spice.SpiceParser.loadTuples(SpiceParser.java:245)
        at edu.anu.spice.SpiceParser.parseCaptions(SpiceParser.java:251)
        at edu.anu.spice.SpiceScorer.scoreBatch(SpiceScorer.java:109)
        at edu.anu.spice.SpiceScorer.main(SpiceScorer.java:60)
Traceback (most recent call last):
  File "h:/thrid_paper/github_code/my_image_captioning_CNN/image_captioning_to_cnn/eval_demo.py", line 53, in <module>
    scorer.compute_scores()
  File "h:/thrid_paper/github_code/my_image_captioning_CNN/image_captioning_to_cnn/eval_demo.py", line 29, in compute_scores
    score, scores = scorer.compute_score(self.gt, self.ref)
  File "h:	hrid_papergithub_codemy_image_captioning_CNNimage_captioning_to_cnncoco_captionpycocoevalcapspicespice.py", line 75, in compute_score
    subprocess.check_call(spice_cmd,
  File "H:	hrid_papergithub_codeimage_captionimage_captioning_envlibsubprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['java', '-jar', '-Xmx8G', 'spice-1.0.jar', 'h:\thrid_paper\github_code\my_image_captioning_CNN\image_captioning_to_cnn\coco_caption\pycocoevalcap\spice\tmp\tmpm4frve7c', '-cache', 'h:\thrid_paper\github_code\my_image_captioning_CNN\image_captioning_to_cnn\coco_caption\pycocoevalcap\spice\cache', '-out', 'h:\thrid_paper\github_code\my_image_captioning_CNN\image_captioning_to_cnn\coco_caption\pycocoevalcap\spice\tmp\tmph6xire8e', '-subset', '-silent']' returned non-zero exit status 1.

==解决办法= =
在spice.py中更改一下代码

        if not os.path.exists(cache_dir):
          os.makedirs(cache_dir)
        spice_cmd = ['java', '-jar', '-Xmx8G', SPICE_JAR, in_file.name,
          # '-cache', cache_dir,
          '-out', out_file.name,
          '-subset',
          '-silent'
        ]