starred/SuiteCRM-SuiteCRM

Fork 0

mirror of https://github.com/SuiteCRM/SuiteCRM.git synced 2026-03-02 19:16:58 -05:00

Not all imported targets are added to target list for large imports #1915

New issue

Closed

opened 2026-02-20 15:17:12 -05:00 by deekerman · 15 comments

deekerman commented

2026-02-20 15:17:12 -05:00

Owner

Originally created by @gitnito on GitHub (May 10, 2017).

Issue

After successfully importing a large number of targets, when you click to "Add to target list", and create a new target list and then select it, only a portion of the imported targets get added to the list. In our last case we imported 7668 targets, but only 2479 got added to the target list. Fortunately we had a way to distinguish the imported targets from previously existing targets, and were able to select them and add them under the targets page. This is a critical bug, since if the targets you imported do not have a lot of unique data it can be difficult or impossible to later separate them to add to target list; and in essence leaving you with thousands of target records on your database that you can not use. They can still be identified directly on the database with the following SQL command:

SELECT * FROM prospects WHERE id NOT IN (SELECT related_id FROM prospect_lists_prospects)

Expected Behavior

All imported targets should be added to the target list.

Actual Behavior

Only a portion of the targets where added to the target list (2479 out of 7668 in our case)

Possible Fix

Maybe there is a limit somewhere. This has happened 2 times to us. The first time we did not realize it until we went to do a campaign and noticed the discrepancy.

Steps to Reproduce

Prepare a Targets.csv file with a large number of records (probably more then 3K but not sure)
Import said file.
After successful import, click "Add to target list" button.
Create a new target list, and select it.
In another window check the number of records added to the new target list.

Context

Importing targets for campaign. In our opinion this is high priority, since it could leave you with thousands of target records in your database not associated with any list; but more importantly, if you are not specifically looking at it, it is very likely you will not notice this issue until days later or after you have performed other imports, further complicating things

Your Environment

SuiteCRM Version used: Version 7.8.3, Sugar Version 6.5.24 (Build 509)
Browser name and version (e.g. Chrome Version 51.0.2704.63 (64-bit)): Firefox 54.0b5 (32-bit)
Environment name and version (e.g. MySQL, PHP 7): mysql Ver 14.14 Distrib 5.5.55, for debian-linux-gnu (x86_64) using readline 6.3, PHP 5.6
Operating System and version (e.g Ubuntu 16.04): Linux 3.16.0-4-amd64 #1 SMP Debian 3.16.39-1+deb8u2 (2017-03-07) x86_64 GNU/Linux

Originally created by @gitnito on GitHub (May 10, 2017).   #### Issue  After successfully importing a large number of targets, when you click to "Add to target list", and create a new target list and then select it, only a portion of the imported targets get added to the list. In our last case we imported 7668 targets, but only 2479 got added to the target list. Fortunately we had a way to distinguish the imported targets from previously existing targets, and were able to select them and add them under the targets page. This is a critical bug, since if the targets you imported do not have a lot of unique data it can be difficult or impossible to later separate them to add to target list; and in essence leaving you with thousands of target records on your database that you can not use. They can still be identified directly on the database with the following SQL command: SELECT * FROM prospects WHERE id NOT IN (SELECT related_id FROM prospect_lists_prospects) #### Expected Behavior  All imported targets should be added to the target list. #### Actual Behavior  Only a portion of the targets where added to the target list (2479 out of 7668 in our case) #### Possible Fix  Maybe there is a limit somewhere. This has happened 2 times to us. The first time we did not realize it until we went to do a campaign and noticed the discrepancy. #### Steps to Reproduce   1. Prepare a Targets.csv file with a large number of records (probably more then 3K but not sure) 2. Import said file. 3. After successful import, click "Add to target list" button. 4. Create a new target list, and select it. 5. In another window check the number of records added to the new target list. #### Context   Importing targets for campaign. In our opinion this is high priority, since it could leave you with thousands of target records in your database not associated with any list; but more importantly, if you are not specifically looking at it, it is very likely you will not notice this issue until days later or after you have performed other imports, further complicating things #### Your Environment  * SuiteCRM Version used: Version 7.8.3, Sugar Version 6.5.24 (Build 509) * Browser name and version (e.g. Chrome Version 51.0.2704.63 (64-bit)): Firefox 54.0b5 (32-bit) * Environment name and version (e.g. MySQL, PHP 7): mysql Ver 14.14 Distrib 5.5.55, for debian-linux-gnu (x86_64) using readline 6.3, PHP 5.6 * Operating System and version (e.g Ubuntu 16.04): Linux 3.16.0-4-amd64 #1 SMP Debian 3.16.39-1+deb8u2 (2017-03-07) x86_64 GNU/Linux

deekerman

2026-02-20 15:17:12 -05:00

closed this issue
added the
Status: Requires Updates
label

deekerman commented

2026-02-20 15:17:14 -05:00

Author

Owner

@pgorod commented on GitHub (May 10, 2017):

Do you see anything in your logs at the time when it's adding to the target list? Maybe it's just the operation that is too long and your PHP configurations block it, or it runs out of memory.

Anyway, it should work, of course, or an error should be shown to the user... but it would be useful to know what you find in suitecrm.log and php_errors.log.

@pgorod commented on GitHub (May 10, 2017): Do you see anything in your logs at the time when it's adding to the target list? Maybe it's just the operation that is too long and your PHP configurations block it, or it runs out of memory. Anyway, it should work, of course, or an error should be shown to the user... but it would be useful to know what you find in `suitecrm.log` and `php_errors.log`.

deekerman commented

2026-02-20 15:17:15 -05:00

Author

Owner

@hemanthvaddi commented on GitHub (Aug 24, 2017):

I have the same issue. Is it solved?

@hemanthvaddi commented on GitHub (Aug 24, 2017): I have the same issue. Is it solved?

deekerman commented

2026-02-20 15:17:15 -05:00

Author

Owner

@pgorod commented on GitHub (Aug 24, 2017):

@hemanthvaddi no, the original poster didn't answer the questions, so this didn't go anywhere. If you have the same problem, and you can answer my questions above, that would be helpful. Also, please tell us your versions of OS, database, web server, PHP and SuiteCRM. Thanks.

@pgorod commented on GitHub (Aug 24, 2017): @hemanthvaddi no, the original poster didn't answer the questions, so this didn't go anywhere. If you have the same problem, and you can answer my questions above, that would be helpful. Also, please tell us your versions of OS, database, web server, PHP and SuiteCRM. Thanks.

deekerman commented

2026-02-20 15:17:15 -05:00

Author

Owner

@hemanthvaddi commented on GitHub (Aug 24, 2017):

OS: Ubuntu 14.04.4
Database: MySQL 14.14
Webserver: Apache 2.4.7
PHP: 5.5.9
SuiteCRM 7.6.4

@hemanthvaddi commented on GitHub (Aug 24, 2017): OS: Ubuntu 14.04.4 Database: MySQL 14.14 Webserver: Apache 2.4.7 PHP: 5.5.9 SuiteCRM 7.6.4

deekerman commented

2026-02-20 15:17:15 -05:00

Author

Owner

@hemanthvaddi commented on GitHub (Aug 24, 2017):

The problem only occurs when I select the list of Targets as a Non-Admin User. As an admin user, it gets all the targets into the TargetList.

The log seems to truncate in the middle of the process. So I'm unable to tail it for longer lists of targets.

@hemanthvaddi commented on GitHub (Aug 24, 2017): The problem only occurs when I select the list of Targets as a Non-Admin User. As an admin user, it gets all the targets into the TargetList. The log seems to truncate in the middle of the process. So I'm unable to tail it for longer lists of targets.

deekerman commented

2026-02-20 15:17:15 -05:00

Author

Owner

@pgorod commented on GitHub (Aug 24, 2017):

Your MySQL version number must be wrong, it should be something like 5.5 or 5.7. But never mind, I don't think that is relevant.

SuiteCRM 7.6.4 is 27 versions ago, there were a LOT of bug fixes meanwhile... I don't know if this could be already fixed.

When tailing logs sometimes the log rotates (gets copied to suitecrm1.log) and the tail goes silent. But if you restart the tail, it keeps going.

Or it could be having a PHP Fatal error that does not get logged in suitecrm.log, but you should see it in php_errors.log.

Without an error message, and on an older system, it's hard to help...

@pgorod commented on GitHub (Aug 24, 2017): Your MySQL version number must be wrong, it should be something like 5.5 or 5.7. But never mind, I don't think that is relevant. SuiteCRM 7.6.4 is 27 versions ago, there were a LOT of bug fixes meanwhile... I don't know if this could be already fixed. When tailing logs sometimes the log rotates (gets copied to `suitecrm1.log`) and the tail goes silent. But if you restart the tail, it keeps going. Or it could be having a PHP Fatal error that does not get logged in `suitecrm.log`, but you should see it in `php_errors.log`. Without an error message, and on an older system, it's hard to help...

deekerman commented

2026-02-20 15:17:15 -05:00

Author

Owner

@hemanthvaddi commented on GitHub (Aug 28, 2017):

I have also checked it in the newer version of SuiteCRM (7.8.3). The issue still seems to persist. I am unable to pursue the source of the issue and why it is affecting only Non-Admin users. In my case when we try to import more than 10000 Targets into a Target List, it only adds 8000-9000 targets.

May I ask if it is possible for you to test this issue from your side as a non-admin user?

@hemanthvaddi commented on GitHub (Aug 28, 2017): I have also checked it in the newer version of SuiteCRM (7.8.3). The issue still seems to persist. I am unable to pursue the source of the issue and why it is affecting only Non-Admin users. In my case when we try to import more than 10000 Targets into a Target List, it only adds 8000-9000 targets. May I ask if it is possible for you to test this issue from your side as a non-admin user?

deekerman commented

2026-02-20 15:17:15 -05:00

Author

Owner

@pgorod commented on GitHub (Aug 28, 2017):

Hi. You can do tests yourself on these two online demos:

demo.suiteondemand.com (only allows regular user access)
www.softaculous.com/demos/SuiteCRM (allows admin access)

You haven't checked your logs yet? You could have a simple error message there telling you directly what the problem is.

@pgorod commented on GitHub (Aug 28, 2017): Hi. You can do tests yourself on these two online demos: demo.suiteondemand.com (only allows regular user access) www.softaculous.com/demos/SuiteCRM (allows admin access) You haven't checked [your logs](https://pgorod.github.io/The-two-logs/) yet? You could have a simple error message there telling you directly what the problem is.

deekerman commented

2026-02-20 15:17:15 -05:00

Author

Owner

@hemanthvaddi commented on GitHub (Aug 29, 2017):

I couldn't test it on demo.suiteondemand because when I try importing a large list of targets, the session is expiring. And this issue isn't there for the admin user, I think there is no point in testing it in softaculous.

@hemanthvaddi commented on GitHub (Aug 29, 2017): I couldn't test it on demo.suiteondemand because when I try importing a large list of targets, the session is expiring. And this issue isn't there for the admin user, I think there is no point in testing it in softaculous.

deekerman commented

2026-02-20 15:17:15 -05:00

Author

Owner

@pgorod commented on GitHub (Aug 29, 2017):

In php.ini, what do you have as memory_limit and max_execution_time? Does increasing those values solve the problem with the large import?

@pgorod commented on GitHub (Aug 29, 2017): In `php.ini`, what do you have as memory_limit and max_execution_time? Does increasing those values solve the problem with the large import?

deekerman commented

2026-02-20 15:17:15 -05:00

Author

Owner

@Dillon-Brown commented on GitHub (Oct 9, 2017):

Closing due to inactivity / inability to replicate. Please reopen if you are able to replicate this as a core issue and provide clear steps. Thanks.

@Dillon-Brown commented on GitHub (Oct 9, 2017): Closing due to inactivity / inability to replicate. Please reopen if you are able to replicate this as a core issue and provide clear steps. Thanks.

deekerman commented

2026-02-20 15:17:15 -05:00

Author

Owner

@rahulner commented on GitHub (Apr 12, 2018):

I was not able to add targets the target list, Search was only adding current page of 20. So i did add via phpmyadmin using below sql , may be useful to someone

insert into prospect_lists_prospects SELECT UUID() id, pl.id prospect_list_id,p.id related_id, 'Prospects' related_type, now() date_modified , 0 deleted FROM prospects p, prospect_lists pl where p.description not like '%your search criteria for targets%' and p.department='another search criteria for target..' and p.deleted = 0 and pl.name = 'Name of the target list' and not EXISTS ( select 1 from prospect_lists_prospects plp where plp.prospect_list_id= pl.id and plp.related_id = p.id )

@rahulner commented on GitHub (Apr 12, 2018): I was not able to add targets the target list, Search was only adding current page of 20. So i did add via phpmyadmin using below sql , may be useful to someone `insert into prospect_lists_prospects SELECT UUID() id, pl.id prospect_list_id,p.id related_id, 'Prospects' related_type, now() date_modified , 0 deleted FROM prospects p, prospect_lists pl where p.description not like '%your search criteria for targets%' and p.department='another search criteria for target..' and p.deleted = 0 and pl.name = 'Name of the target list' and not EXISTS ( select 1 from prospect_lists_prospects plp where plp.prospect_list_id= pl.id and plp.related_id = p.id ) `

deekerman commented

2026-02-20 15:17:15 -05:00

Author

Owner

@lucas7793 commented on GitHub (May 28, 2018):

I'm experiencing the same problem in the current version 7.10.5. I created a report with 4579 results. Then I click on Add to Prospect List. After it has finished there are only 3136 entries in the selected prospect list. But it's not that the list is cut of somewhere, there are missing entries in between. After trying to add the prospects again to the same list, the amount does not change. From the logs I can not see any errors.
For me it also does not change if it's a admin or non-admin user.

@lucas7793 commented on GitHub (May 28, 2018): I'm experiencing the same problem in the current version 7.10.5. I created a report with 4579 results. Then I click on `Add to Prospect List`. After it has finished there are only 3136 entries in the selected prospect list. But it's not that the list is cut of somewhere, there are missing entries in between. After trying to add the prospects again to the same list, the amount does not change. From the logs I can not see any errors. For me it also does not change if it's a admin or non-admin user.

deekerman commented

2026-02-20 15:17:16 -05:00

Author

Owner

@pgorod commented on GitHub (May 28, 2018):

@lucas7793 that is a good issue to raise in the forums, not here. I will help you out there.

You need to look at your logs and report any errors there at the time of the failed import.

@pgorod commented on GitHub (May 28, 2018): @lucas7793 that is a good issue to raise in the forums, not here. I will help you out there. You need to look at your logs and report any errors there at the time of the failed import.

deekerman commented

2026-02-20 15:17:16 -05:00

Author

Owner

@dtosun61 commented on GitHub (Aug 15, 2018):

I'm experiencing the same problem in the my version 7.8.18. Any solution for this?

@dtosun61 commented on GitHub (Aug 15, 2018): I'm experiencing the same problem in the my version 7.8.18. Any solution for this?