Hi Brooks
What is the processing actually doing? You say it adds "a couple thousand records" to the smaller back-end, but it would help to know what is being read from the large, static back-end, and how the table(s) are queried.
If you have code like this:
1. open a recordset on back-end A with a WHERE clause and read some values
2. execute an INSERT INTO query on back-end B
3. close the recordset on back-end A
4. repeat a couple of thousand times
---In MS_Access_Professionals@yahoogroups.com, <dp_midknight@...> wrote :
then this will be VERY slow, unless you have a persistent connection open to both backends, because each iteration will need to open and close both files.
Are you reading from the single table in back-end A, or are your reading from a SQL query comprising multiple JOINed tables? If it is a single table, then you will benefit by opening the database and then opening the table within that database (not the linked table) as a table-type recordset. Then you can use the Seek method to find the target record.
If it is a SQL string with JOINs, then you might benefit by saving the query as a parameter query in the back-end. Then you can open the QueryDef in the external databese, fill the parameters, and execute an OpenRecordset on the QueryDef.
I agree with Crystal that a single long-integer primary key will perform better than a composite primary key, especially if it is used in JOINs. You can keep the unique composite key, but make it a secondary key.
I also agree with Stuart and Brian, that converting it to SQL Express would give the best benefit, but that could be a lot of work and I suggest you try some of the other suggestions first.
Best wishes,
Graham [Access MVP, 1996-2016]
---In MS_Access_Professionals@yahoogroups.com, <dp_midknight@...> wrote :
I agree with Stuart.. If you converted it to MSSQL, or even MYSQL, You could execute these SQL as pass-through, and have the SQL Engine which is far better at huge datasets like this and I"m thinking you could trim that down to say, less than an hour? Maybe a few minutes?
-Brian
On Tuesday, December 4, 2018, 5:21:50 PM EST, Stuart Schulman stoughy@... [MS_Access_Professionals] <MS_Access_Professionals@yahoogroups.com> wrote:
Access is not all that great when the tables get this large. Would suggest moving to SQL and stored procedures.
Stuart
On Tuesday, December 4, 2018, 4:56:02 PM EST, crystal 8 strive4peace2008@... [MS_Access_Professionals] <MS_Access_Professionals@yahoogroups.com> wrote:
primary keys is not necessary for AutoNumbers -- just unique index
On 12/4/2018 3:54 PM, crystal 8 wrote:
hi Brooks,
I would suggest adding AutoNumber primary keys (with unique index) and Long Integer Foreign keys (default value=Null, index that allows duplicates and ignores null) with a set of queries to update the foreign keys. Then use the Long Integers to link. Add fields to the end if you import and other fields need to be in a certain order. This will greatly enhance performance.Size:
AutoNumber/Long Integer = 4 byte
2 dates, a long number and a 1 character text fields = 2*8 + 4 + ? --> more than 20So just by doing this, each join will be at least 5 times faster. The indexes will help speed it up too.
~crystal
On 12/4/2018 3:14 PM, brooksrimes@... [MS_Access_Professionals] wrote:Hi Crystal,Thanks for the post.Unfortunately, the 3 million row tables needs 4 fields for the primary key: 2 dates, a long number and a 1 character text fields.The frontend and backend are both on my Dell laptop with a Intel Core i5 processor and 8 GB of RAM. I just upgraded from a HDD to a SSD and there was only a 9% reduction in run time.I compact and repair regularly. I've been using Access since 1.0.A separate database server running SQL Server is probably an option.Brooks
__._,_.___
Posted by: graham@mandeno.com
| Reply via web post | • | Reply to sender | • | Reply to group | • | Start a New Topic | • | Messages in this topic (8) |
Have you tried the highest rated email app?
With 4.5 stars in iTunes, the Yahoo Mail app is the highest rated email app on the market. What are you waiting for? Now you can access all your inboxes (Gmail, Outlook, AOL and more) in one place. Never delete an email again with 1000GB of free cloud storage.
SPONSORED LINKS
.
__,_._,___
Tidak ada komentar:
Posting Komentar