mirror of
https://github.com/SuiteCRM/SuiteCRM.git
synced 2026-03-02 19:16:58 -05:00
Scheduler AOD/LUCENE index problems #630
Labels
No labels
Area: API
Area: Campaigns
Area: Cases
Area: Clean Up
Area: Clean Up: Performance
Area: Dashlets
Area: Databases
Area: Developer Tools
Area: Elasticsearch
Area: Elasticsearch
Area: Emails
Area: Emails:Campaigns
Area: Emails:Cases
Area: Emails:Compose
Area: Emails:Config
Area: Emails:Templates
Area: Environment
Area: Installation
Area: Language
Area: Mobile
Area: Module
Area: PDFs
Area: PHP8
Area: Reports
Area: Studio
Area: Styling
Area: Upgrading
Area: Workflow
Area:Activity Stream
Area:Calls
Area:Import
Area:Projects
Area:Search
Area:Surveys
Area:Themes
Area:Users
Branch:Hotfix
Good First Issue
Hacktoberfest
Help Wanted
PR:Community Contribution
PR:Type:Enhancement
Priority:Critical
Priority:Important
Priority:Moderate
Severity: Major
Severity: Minor
Severity: Moderate
Status: Requires Code Review
Status: Requires Updates
Status: Stale
Status: Team Investigating
Status:Assessed
Status:Fix Proposed
Status:Needs Assessed
Status:Requires Automated Tests
Type: Bug
Type:Deprecated
Type:Discussion
Type:Duplicate
Type:Invalid
Type:Question
Type:Suggestion
Type:Suggestion
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/SuiteCRM-SuiteCRM#630
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @adamjakab on GitHub (Feb 25, 2016).
I'm not sure what these are for and for now I have disabled them but I cannot run cron with these enabled:
php cron.php
PHP Fatal error: Uncaught exception 'Zend_Search_Lucene_Exception' with message 'Index doesn't exists in the specified directory.' in /home/crm2.ingrossomaterialipulizia.it/httpdocs/modules/AOD_Index/Lib/Zend/Search/Lucene.php:566
Stack trace:
#0 /home/crm2.ingrossomaterialipulizia.it/httpdocs/modules/AOD_Index/AOD_Index.php(318): Zend_Search_Lucene->__construct('modules/AOD_Ind...')
#1 /home/crm2.ingrossomaterialipulizia.it/httpdocs/modules/AOD_Index/AOD_Index.php(68): AOD_Index->getLuceneIndex()
#2 /home/crm2.ingrossomaterialipulizia.it/httpdocs/modules/Schedulers/_AddJobsHere.php(678): AOD_Index->optimise()
#3 [internal function]: aodOptimiseIndex(Object(SchedulersJob))
#4 /home/crm2.ingrossomaterialipulizia.it/httpdocs/modules/SchedulersJobs/SchedulersJob.php(522): call_user_func_array('aodOptimiseInde...', Array)
#5 /home/crm2.ingrossomaterialipulizia.it/httpdocs/include/SugarQueue/SugarCronJobs.php(188): SchedulersJob->runJob()
#6 /home/crm2.ingrossomaterialipulizia.it/httpdocs/include/SugarQueue/SugarCronJobs.php(228): in /home/crm2.ingrossomaterialipulizia.it/httpdocs/modules/AOD_Index/Lib/Zend/Search/Lucene.php on line 566
@mrbarletta commented on GitHub (Feb 27, 2016):
Got this issue after an upgrade from a previous version.
Check you have this directory
modules/AOD_Index/Index/Index/content should be like this:
`-rw-rw-rw- 1 www-data www-data 0 Feb 27 03:00 optimization.lock.file
-rw-rw-rw- 1 www-data www-data 0 Feb 27 15:01 read.lock.file
-rw-rw-rw- 1 www-data www-data 0 Feb 27 03:00 read-lock-processing.lock.file
-rw-rw-rw- 1 root root 43 Feb 27 03:00 segments_1uc
-rwxrwxr-x 1 www-data www-data 20 Feb 27 03:00 segments.gen*
-rw-rw-rw- 1 www-data www-data 0 Feb 27 03:00 write.lock.file
`
In my case I was missing some 0 sized files, i created them and voilà!
@dpavlovic commented on GitHub (Jan 26, 2017):
I have this log/problem during upgrade procedure (using wizard) 7.6.10 -> 7.7.9.
26-Jan-2017 05:07:01 America/New_York] PHP Fatal error: Uncaught exception 'Zend_Search_Lucene_Exception' with message 'Index doesn't exists in the specified directory.' in /home/xxxxxxxxxxx/public_html/modules/AOD_Index/Lib/Zend/Search/Lucene.php:566 Stack trace: #0 /home/xxxxxxxxxxx/public_html/modules/AOD_Index/AOD_Index.php(342): Zend_Search_Lucene->__construct('modules/AOD_Ind...') #1 /home/xxxxxxxxxxx/public_html/modules/AOD_Index/AOD_Index.php(83): AOD_Index->getLuceneIndex() #2 /home/xxxxxxxxxxx/public_html/modules/Schedulers/_AddJobsHere.php(683): AOD_Index->optimise() #3 [internal function]: aodOptimiseIndex(Object(SchedulersJob)) #4 /home/xxxxxxxxxxx/public_html/modules/SchedulersJobs/SchedulersJob.php(536): call_user_func_array('aodOptimiseInde...', Array) #5 /home/xxxxxxxxxxx/public_html/include/SugarQueue/SugarCronJobs.php(188): SchedulersJob->runJob() #6 /home/xxxxxxxxxxx/public_html/include/SugarQueue/SugarCronJobs.php(228): SugarCronJobs->executeJob(Object(SchedulersJob)) #7 /home/xxxxxxxxxxx/public_html/cron.php(74): Sugar in /home/xxxxxxxxxxx/public_html/modules/AOD_Index/Lib/Zend/Search/Lucene.php on line 566@pgorod commented on GitHub (Feb 14, 2017):
Have any of you tried to recreate the Index from scratch? I'm curious if the problems would reappear or not. I would bet these are all just inconsistencies in the Index. They might be Zend bugs, not SuiteCRM bugs.
Although I believe this issue #2788 would need to be solved before we can really acquit SuiteCRM from messy installations found after upgrades... in the current state of things, it's only natural that upgrades will mess things up occasionally, you just need a bit of bad luck.
@pgorod commented on GitHub (May 2, 2017):
@samus-aran based on my experience in the forums, I would say this issue could be closed as it is somewhat common to see broken Indexes that need rebuilding. This could be caused by Lucene bugs, by degraded permissions (#2787), or by unfortunate interactions of cron jobs with upgrade processes (#2788).
It would be useful to have a bug report where the corruption event could be reproduced reliably, but that doesn't seem to be the case with this Issue, and it's been quite inactive for a long time.
Of course this is just a suggestion and if any of the people in the thread object, I have no problem with keeping this open.
@chris001 commented on GitHub (May 3, 2017):
@pgorod I respectfully have to not agree with you on this one. This issue is truly a bug which kills the user experience and should stay open until it is fixed.
@pgorod commented on GitHub (May 4, 2017):
@chris001 I agree with all you wrote, this needs to work! And SuiteCRM badly needs solving all these tripping points.
What I meant was: this is a frequent problem; as far as I can tell, the causes are probably elsewhere, in other Issues already being dealt with. I think for most cases the corruption comes from wither the cron jobs running as a different user (already mitigated in 7.8.3) or the cron jobs running during maintenance and upgrade (#2788); or maybe there are bugs in Lucene, but it seems we should be dropping Lucene in favor of another library that is still maintained. So...
What I mean is I can't extract anything useful from this particular Issue to help me debug. I would need a reproducible case of the corruption being produced, not just the verification that it has occurred in the past. But I do agree this needs to "just work"!
@chris001 commented on GitHub (May 4, 2017):
They're getting the bug when upgrading 7.6 to 7.7.9.
@pgorod commented on GitHub (May 4, 2017):
I believe it's much more random than that, it only happens on a few occasions; I think it's an interaction between two processes: the
AOD Optimize Indexcron job (which is throwing the error) and the installer process.I'm afraid I don't have time now to go into this sort of investigation... but even if I had, I would prefer to devote it to #2788, I believe it would solve this issue and a bunch of others. :-)
@chris001 commented on GitHub (May 4, 2017):
This bug is one which can and should be tested automatically by
phpunittests run by Travis or by the command line. Travis/phpunit/command line is already able to silently install the app. Silent upgrade is practically the same thing.@jyotiraghav93 commented on GitHub (May 25, 2018):
I am also getting the same issue, Uncaught exception 'Zend_Search_Lucene_Exception' , any luck?
@pgorod commented on GitHub (May 25, 2018):
https://pgorod.github.io/Reindex-AOD/
@pgorod commented on GitHub (May 25, 2018):
@Dillon-Brown I would close this issue, even though this code is obviously lacking something, there is nothing in this issue that helps us debug. There's no point in keeping this open forever. Now that we are beginning to have proper tests we should be getting closer to the basic frailties of the indexing mechanism.
@jyotiraghav93 commented on GitHub (May 25, 2018):
Asunto: Cron webdesi0@siteground235 cd /home/webdesi0/public_html/suitecrm; php -f cron.php
Fecha: Thu, 24 May 2018 14:05:02 -0500
De:
Cron Daemon webdesi0@siteground235.com
Para: playawebdesign@gmail.com
PHP Fatal error: Uncaught exception 'Zend_Search_Lucene_Exception' in /home/webdesi0/public_html/suitecrm/modules/AOD_Index/Lib/Zend/Search/Lucene/Storage/File/Filesystem.php:65
Stack trace:
#0 /home/webdesi0/public_html/suitecrm/modules/AOD_Index/Lib/Zend/Search/Lucene/Storage/Directory/Filesystem.php(359): Zend_Search_Lucene_Storage_File_Filesystem->__construct('modules/AOD_Ind...')
#1 /home/webdesi0/public_html/suitecrm/modules/AOD_Index/Lib/Zend/Search/Lucene/Index/SegmentInfo.php(286): Zend_Search_Lucene_Storage_Directory_Filesystem->getFileObject('_h.cfs')
#2 /home/webdesi0/public_html/suitecrm/modules/AOD_Index/Lib/Zend/Search/Lucene.php(495): Zend_Search_Lucene_Index_SegmentInfo->__construct(Object(Zend_Search_Lucene_Storage_Directory_Filesystem), '_h', 1, -1, NULL, 1, true)
#3 /home/webdesi0/public_html/suitecrm/modules/AOD_Index/Lib/Zend/Search/Lucene.php(570): Zend_Search_Lucene->_readSegmentsFile()
#4 /home/webdesi0/public_html/suitecrm/modules/AOD_Index/AOD_Index.php(342): Zend_Search_Lucene->__construct('modules in /home/webdesi0/public_html/suitecrm/modules/AOD_Index/Lib/Zend/Search/Lucene/Storage/File/Filesystem.php on line 65
Fatal error: Uncaught exception 'Zend_Search_Lucene_Exception' in /home/webdesi0/public_html/suitecrm/modules/AOD_Index/Lib/Zend/Search/Lucene/Storage/File/Filesystem.php:65
Stack trace:
#0 /home/webdesi0/public_html/suitecrm/modules/AOD_Index/Lib/Zend/Search/Lucene/Storage/Directory/Filesystem.php(359): Zend_Search_Lucene_Storage_File_Filesystem->__construct('modules/AOD_Ind...')
#1 /home/webdesi0/public_html/suitecrm/modules/AOD_Index/Lib/Zend/Search/Lucene/Index/SegmentInfo.php(286): Zend_Search_Lucene_Storage_Directory_Filesystem->getFileObject('_h.cfs')
#2 /home/webdesi0/public_html/suitecrm/modules/AOD_Index/Lib/Zend/Search/Lucene.php(495): Zend_Search_Lucene_Index_SegmentInfo->__construct(Object(Zend_Search_Lucene_Storage_Directory_Filesystem), '_h', 1, -1, NULL, 1, true)
#3 /home/webdesi0/public_html/suitecrm/modules/AOD_Index/Lib/Zend/Search/Lucene.php(570): Zend_Search_Lucene->_readSegmentsFile()
#4 /home/webdesi0/public_html/suitecrm/modules/AOD_Index/AOD_Index.php(342): Zend_Search_Lucene->__construct('modules in /home/webdesi0/public_html/suitecrm/modules/AOD_Index/Lib/Zend/Search/Lucene/Storage/File/Filesystem.php on line 65
this issue is not fixed through your given link https://pgorod.github.io/Reindex-AOD/ already tried it
@pgorod commented on GitHub (May 25, 2018):
Those errors are not the same as the one previously mentioned in the issue. That's why I feel this should be handled in the Forums, not here. Here we need to be able to move beyond individual, disparate user reports; we needs stuff that is reproducible or that, at least, points to some specific area of the code for us to look into. IMHO :-)
@jyotiraghav93 commented on GitHub (May 25, 2018):
Okay thanks
@Dillon-Brown commented on GitHub (May 25, 2018):
I'm not able to replicate this issue so I agree we should take this to the forums and treat it as individual user issues until someone can reliably replicate this as a bug that we can investigate.